Peer Review and the Productivity Era

Mark Bauerlein

In its ideal version, peer review in the humanities operates like this. A body of work is presented to accredited experts who examine it on proper disciplinary grounds and render a judgment. The body of work may be a book manuscript submitted to a scholarly press for publication, or a thick dossier of a candidate for tenure comprised of teaching, service, and research materials, or a writing sample requested from an applicant for a tenure-track position, or a grant or fellowship application that contains a detailed description of a research project to be completed during the term of the award.

The accredited figures may be a five- or three-person hiring committee working through a stack of job applications to select ten for an interview at a professional convention, then to identify two or three for a campus visit. Or, it may be two established scholars in a subfield to whom an academic press editor sends a manuscript and who respond with scrupulous reader’s reports. Or, it could be the entire tenured faculty of a department voting yes or no on the promotion of an assistant professor.

And the proper disciplinary criteria? In the humanities, peer review involves a mixture of empirical, logical, field-specific, and stylistic factors. Every statement of fact must be accurate, the thesis must be distinct, the arguments cogent (with relevant and sufficient evidence on hand), and the interpretations plausible. Knowledge of the field should be obvious and sharply-wielded, and the prose must be readable. (Don’t dismiss the stylistic demand, for academic prose in the humanities is often awkward and dense, self-important and confounding.) The reviewers should be objective—no personal or political biases allowed. If a writing sample develops an interpretation that surprises or annoys reviewers, but the transitions are tight, the evidence substantive, and the language limpid and lively, then reviewers must approve it.

Here is how it worked once several years ago. At the time, I read manuscripts for many presses and quarterlies, the editors apparently favoring my willingness to descend to the level of line editing in my evaluations. (Often irritated by the clunky, clotted prose style of the submissions, I would select a representative paragraph or two from the text, rewrite it from beginning to end, and then advise the author to do the same throughout the whole thing.) I also offered extensive developmental editing advice, taking a manuscript’s design on its own terms and explaining how the structure and focus of the chapters needed to change in order to realize it.

Editors liked the response, I think, because it gave them an answer to the authors that didn’t mean acceptance or rejection. They knew the manuscript wasn’t suitable for publication, but instead of giving a blunt “No,” they could suspend their decision while supplying constructive guidance. My own report typically began not with “Let me enumerate the many drawbacks of this manuscript,” but rather with “I am happy to assess _____ for the Press, but having read through it I conclude that it’s not quite ready for scholarly evaluation.” Scholarly evaluation couldn’t happen until some specific issues of conception and composition had been addressed. Given the often high stakes of the editor’s communication (the author’s job might depend on it), a reader’s report that was both forward-looking and exacting—betraying nothing personal or ideological in the evaluation—gave editors a way to say “Not yet,” as if their lack of a “Yes” response were in the service of bringing the manuscript to fruition.

This was the context in which an editor contacted me about reviewing a manuscript on American poetry. The press was a hot one at the time, collecting well-known authors in cultural studies and gender theory, as well as publishing edgy volumes in American studies. I agreed and the manuscript arrived a few days later. The editor had noted that he already had two reader’s reports on the manuscript but required another. I assumed he needed me because the previous two reports disagreed starkly over the quality of the manuscript and a third report would decide its fate.

I looked at them first. To my surprise, the two reader’s reports agreed 100 percent. More than that, they urged publication by the press directly, suggesting no revisions at all before the work went into production. Indeed, both recommendations went over the top, heralding the manuscript as a brilliant and far-reaching contribution to the field. They spoke glowingly, raising their comments to the level usually reserved for groundbreaking works that inspire a thousand imitations. I don’t recall a whisper of criticism, constructive or otherwise, nor even any specific praise such as noting how the author’s chapter on this or that poet revised settled thinking about this or that poem. Instead, the summaries were broad and windy, as if the recommenders’ praise needed nothing more to authorize them than the signatures at the bottom of the reports. I checked each one and recognized the names as famed figures in the Modern Language Association world, one of them prominent in the New York literary scene, too.

I turned to the manuscript to see what had provoked the effusions. Two hours of examination didn’t illuminate the virtues proclaimed in the reader’s reports. On the contrary, the manuscript appeared more or less the same as dozens of literary studies that come out each year from the scholarly presses. It bore some of the typical problems, too; for instance, the manuscript ran for two hundred pages longer than it should have, given the nature of the project—an interpretation of modern American poetry in light of a particular topic. It contained little archival material (which can swell a monograph), and the opening devoted too many pages to well-worn trends in literary studies in the preceding decades, the sign of a junior scholar trying too hard to demonstrate his involvement in the field.

Others revisions were needed, but nothing out of the ordinary, and the author displayed enough knowledge and intelligence to lead me to suggest alterations that, if followed, would clear it for publication. Whatever its merits and deficiencies, however, it didn’t come close to matching the statements in the reader’s reports. Indeed, the extraordinary acclaim of the letters appeared downright suspect as soon as I began poring through the manuscript and found it a customary effort. The discrepancy explained why the editor wanted a third reader. He knew that he could not go to his editorial board with two evaluations that patently amounted to little more than puffery. Their overstatements required some tempering by an analytical report. I proceeded to provide it, but not before investigating why two distinguished literature professors who’d written superb books should have so mis-estimated this one.

It didn’t take long. I looked up the author and found that he was currently working with the first reader on a different endeavor. Furthermore, their association dated from years earlier, for the author was a graduate student of the first reader, and this manuscript was a version of the dissertation directed and signed by the first reader! The second reader was a close colleague of the first (once at an invitation-only academic meeting I heard reader #2 rise in the audience solemnly to defend #1 after a speaker had knocked him). Added to those conflicts of interests was the fact that the manuscript’s author was coming up for tenure and needed a book contract to survive.

The book was published a year later and the author graduated to associate professor, my final reaction being a shake of the head in disgust at the whole episode. What utter corruption, I thought at the time—a cheap inside job, a clear sign that incompetence and cynicism had overtaken my field at the highest levels. By that point in my career I had witnessed firsthand so many cases of shoddy and fraudulent peer review—and heard of so many others—that the enterprise of the humanities seemed compromised. The infamous Sokal hoax of a couple years before was only the most publicized example, one embarrassing breakdown that implied a thousand similar lapses that never came out. Of course, rigorous peer review still happened, and most of the evaluators in the humanities perhaps still approached their task with integrity, but to my observation the occasions of ineptitude and corner-cutting had reached a critical mass. There were too many to pass them off as the ordinary lapses every profession bears.

Today, however, I no longer interpret the episode in that way. Instead of judging it as a case of cronyism and sloppiness, I see it as a recognition of the changed status of humanities research. The editor and readers were not shady and bungling. Rather, they discerned quite clearly the stakes of peer review in American literary studies, more clearly than I had. I wanted to apply rigorous scholarly criteria to the manuscript. Why? Because I believed that the research enterprise needed stewardship, vigilant protection against mediocre, false, and inelegant thoughts and words. I assumed that the corpus of literary scholarship relied on stern gatekeepers, and that aspiring scholars needed to be disciplined into it, their works sharpened and refined before they could enter the tradition of Lionel Trilling, Cleanth Brooks et al.

But where was literary studies in the late-1990s? It was already overrun by thoughts and ideas of all kinds, every field and subfield saturated with new theories, added interpretations, more revisionist ideas. The great literary studies of the past such as I. A. Richards’s Practical Criticism (1929) and Erich Auerbach’s Mimesis (1946) had already faded into oblivion—graduate students and recent Ph.D.s never having heard of them. The profession had changed, and the critical heritage had faded. And it wasn’t because of the takeover by trendy and tendentious notions such as trauma theory and postcolonialism. That intellectual decay, in fact, was an epiphenomenon. The root cause was simpler and more material than that, a trend fully captured numerically. Literary studies circa 1999 was crumbling under the sheer volume of books, articles, and reviews published every year, and the institutional memory of criticism of the past couldn’t survive it. Nor could rigorous peer review.

The accumulation was staggering, and it had been climbing for decades. In the 1950s, the number of items of scholarship published each year in all the languages and literatures was in the low ten-thousands. Now, it exceeds 70,000.1 The Modern Language Association listed 1,139 periodicals publishing literary scholarship in 1959, while today it counts 4,600.2 By the turn of the century, the impact was firm: The proliferation of books and articles on major authors and movements and genres had grown so large that it rendered every new book and article in each area ever less significant. Here are some examples.

From 1930 to 1960, only eleven books were published containing significant material (at least a chapter) on Charlotte Brontë. At that rate, every new volume on Brontë from a distinguished press marked an important step in the study of the author. But from 1961 to 1999, the MLA Bibliography records 257 such books.

For William Wordsworth, the numbers are 110 books from 1930 to 1960, an average of less than four books per year—a rate at which Romantic scholars around the country could glance through every book published on the poet and maintain universal knowledge of the critical record through their entire careers. Each effort would receive a full reception. From 1961 to 1999 the tally leaped to 804 books, an indigestible amount. (Keep in mind that the book total doesn’t include all the essays, editions, dissertations, conference papers, and reviews published on Wordsworth.) At that rate of issue, ten or twenty books per year might go effectively unnoticed, the sheer bulk of pages leading everyone to form a different relationship to their field. No longer could scholars read everything that came out, and as the years passed by and nothing terrible happened to them, they recognized that they didn’t have to read everything published on their field.

What else did this explosion signify? Literary scholars immersed in the system consider the expansion of scholarship a sign of enthusiasm and fruitfulness, and perhaps they have a point. But the impact upon peer review was constricting. Evaluators with any familiarity with publications in the field couldn’t avoid a worrisome question. With 1,858 items of scholarship published on American poetry in the last three decades of the twentieth century, was it realistic to think that the 1,859th item would add all that much to the critical record? If not, was it really necessary to demand so meticulous a peer review, to spend time and energy leading an author to revise a manuscript from “passable” to “excellent”?

With so many other books out there, this one probably wouldn’t attract the discriminating attention that would even register the difference. Besides, if another ho-hum book got published, nobody would get hurt, and if it didn’t get published a career might end. So why be such a fussbudget? It would have its costs, to be sure—for instance, the research pay a professor collects while writing it, the money libraries spend to purchase it—but those expenses would be so mediated that people could ignore them. On the other hand, livelihoods might be tied to the publication of that middling manuscript, opening a troubling disjunction between the ultra-high stakes for the author and the low stakes for the discipline. Once again, given that inequality, why be a peer-review prig?

The customary answer to the question at that time was, “For the purposes of quality control,” an answer that assumed a certain standing and import of the field. But that answer no longer obtains. While in the sciences faulty research has critical consequences that can do lasting damage to the field and to everybody who relies upon it, in the humanities no such impact takes place. Poor scholarship simply disappears. Yes, we have seen certain glib, hackneyed, and outright wrong books spawn a hundred others, but not for many years now. The last one, I think, was Michael Hardt and Antonio Negri’s Empire (2000), and it was such a rarity that it sparked an unintentionally whimsical New York Times story entitled, “What Is the Next Big Idea? The Buzz Is Growing.”3

The Times article itself was a fair measure of how the scholarly norms in the humanities had shifted in the preceding twenty years. Instead of talking about Empire in terms of truth, knowledge, and discovery, author Emily Eakin cast it as “hot,” a “trendy subject,” “an impact,” and “filling a void in the humanities.” It mentioned the book’s “new way of thinking about global politics,” an element that one would assume marks its highest import, yet Eakin said nothing more specific about it. Instead, she pondered whether Empire would have “staying power” and whether Hardt would be the next Jacques Derrida.4 The book’s place in the academic scene counted more than the content of its propositions.

One couldn’t imagine talking about M. H. Abrams and The Mirror and the Lamp in the same way when the book appeared in 1953, or about Paul de Man once Allegories of Reading was published in 1979. The jargon of “currency” hadn’t ascended yet, and if it had, both men would have regarded the discussion as embarrassingly vulgar. But when so many books and essays are published each year, one can’t avoid it. The amount of attention each one receives becomes a significant element in academic conversation—not just the internal virtues and vices of the work as detailed by peer review, but its outer noteworthiness as determined by the number of people talking about it. Here, in other words, we have another effect of gushing research productivity in the humanities, another displacement of peer review criteria.

It can’t go on. If a field isn’t lively and consequential enough to solicit exacting peer reviews 90 percent of the time, then the research enterprise isn’t worth it. People start to notice quality control issues in the results (as happened in the humanities starting in the 1980s). If the most influential figures in the field decide to lower the bar at selected moments, then the endeavor begins to show symptoms of decline. Those reviewers who do soften their judgment aren’t causal factors, though. They merely respond to an unworkable circumstance—the glut of scholarly publication—and whatever epistemological principles they had twenty-five years ago cannot withstand the material pressures of today. The decline of peer review in the humanities will not end unless the amount of material published each year contracts sharply, and I see no evidence that that sane and sensible development is going to happen.

  • Share
Most Commented

April 16, 2021


Social Justice 101: Intro. to Cancel Culture

Understanding the illogical origin of cancel culture, we can more easily accept mistakes, flaws, and errors in history, and in ourselves, as part of our fallen nature....

April 19, 2021


Critical Race Theory and the Will to Power

A review of "1620: A Critical Response to the 1619 Project" by NAS President Peter W. Wood....

May 30, 2018


The Case for Colonialism

From the summer issue of Academic Questions, we reprint the controversial article, "The Case for Colonialism." ...

Most Read

August 23, 2021


Testing the Tests for Racism

What is the veracity of "audit" studies, conducted primarily by sociologists, that appear to demonstrate that people of color confront intense bias at every level of society?...

May 30, 2018


The Case for Colonialism

From the summer issue of Academic Questions, we reprint the controversial article, "The Case for Colonialism." ...

March 29, 2019


Homogenous: The Political Affiliations of Elite Liberal Arts College Faculty

A study on the partisanship of liberal arts professors at America's top universities. ...