This Column Is So Fake

A few MIT Computer Science graduate students with nothing better to do decided it might be fun to develop a system that automatically generates bogus Computer Science papers-complete with graphs, diagrams and fake citations. I decided to try it out for myself-now I've got a PDF file of a paper that I wrote with Thomas Alva Edison and Albert Einstein called "The Effect of Game-Theoretic Communication on Cryptography."

The system, called SCIgen, has become increasingly popular since its inception, gaining publicity through the viral marketing of spam and AIM profiles. But the notoriety of SCIgen and its creators has exploded in the past few weeks and played a key role in discussions about the quality of the atmosphere in academia. There's nothing wrong with SCIgen itself-the system is actually a very interesting study in grammar constructions and sentence meaning. The real problem lies in the acceptance of bogus papers at seemingly legitimate conferences. Yes, that's right: One of SCIgen's automatically generated papers was recently accepted to the World Multiconference on Systemics, Cybernetics and Informatics (WMSCI), to be held in Orlando in July.

Here's a paragraph from the first section of the accepted paper:

Certainly, the usual methods for the emulation of Smalltalk that paved the way for the investigation of rasterization do not apply in this area. In the opinions of many, despite the fact that conventional wisdom states that this grand challenge is continuously answered by the study of access points, we believe that a different solution is necessary. It should be noted that Rooter runs in O(log log n) time. Certainly, the shortcoming of this type of solution, however, is that compilers and superpages are mostly incompatible. Despite the fact that similar methodologies visualize XML, we surmount this issue without synthesizing distributed archetypes.

Now to most people, computer science papers look like gibberish anyway, but even a casual reader can recognize the gaping inconsistencies with noun-pronoun and subject-verb agreement. And anyone who passed reading comprehension should recognize that there are almost no logical connections between one sentence and the next. The existence of sentences like, "As a result, we conclude." (which exist in every paper the system generates) might also clue one into the dubious nature of these papers.

To the technically or even jargon-savvy academic, this should be even more obvious. This paragraph makes almost no real sense whatsoever-what do compilers, superpages and XML have to do with one another in this context? What the hell is a superpage, anyway?

Most of this paper seems to recall linguist Noam Chomsky's classic quest to find sentences that are correct grammatically but make no sense in any context and are otherwise impossible. Chomsky came up with "Colorless green ideas sleep furiously." By comparison, SCIgen didn't do so bad for itself: "Combined with relational archetypes, this synthesizes new wearable algorithms." Wearable algorithms?

How did a paper so obviously devoid of any logic end up being accepted to the conference?

The official party line is that the paper had not been reviewed before an internal deadline, and the organizers felt it would be unfair to exclude papers that had not been officially rejected after review. The conference's Web site includes a lengthy section titled "With regards to the bogus papers submitted to WMSCI 2005," that begins by emphasizing the status of the accepted submission as a "NONREVIEWED paper."

However, the text goes on to cite numerous papers that defend its lax acceptance policy-and even support the admission of nonreviewed papers and papers that receive poor feedback. Ignoring the irony in defending the acceptance of bogus papers by citing other random papers, there ought to be some basic standard for admitting papers to a conference.

BBC News reports that Professor Nagib Callaos, the conference's general chair, is reviewing the conference's acceptance procedures. Well, there's a start. I have a suggestion for Professor Callaos, as he redevelops the standards for acceptance to WMSCI: Consider coherency. I mean, come on-at least get someone to read all the papers. As a result, we conclude.

Tags: