The High Quality Research Act: A Blast from the Past?

Melinda Baldwin, a historian of science interested in the development of peer review, has written a guest post about some interesting parallels between the High Quality Research Act and an older controversy about peer review at the NSF. You can learn more about her work here.

A few months ago, Hank and Lee shared some thoughts about the discussion surrounding the "High Quality Research Act," a bill drafted by Rep. Lamar Smith (R-TX), the current head of the House Committee on Science, Space and Technology. The bill would require the NSF director to pledge that funded projects are "high-quality" and benefit the American people, and it seems to be grounded in Smith's concern that the NSF is funding "questionable" projects. Shortly before a draft of the HQRA leaked, Smith had called Presidential science advisor John Holdren and acting NSF Director Cora Marrett before Congress to justify the NSF's spending decisions. 

Smith's repeated statement that he wanted to "improve" on the NSF's grant-awarding process raised hackles in the scientific community. Currently, the NSF relies on reports from referees—i.e., peer review—to choose which applications will be funded. What many observers found really alarming was the letter Smith wrote to Marrett, requesting copies of the referee reports related to five NSF grants that he felt were suspicious, all in the social sciences. 

http://www.nsf.gov/news/mmg/media/images/nsf_bldg_f1.jpg

Reaction from the scientific community, and from politicians, was swift. Most of it played on the same theme: that peer review is a sacrosanct part of the scientific process and that Congressional interference would have dire consequences for the quality of research in the United States. Eighteen former NSF Assistant Directors signed a letter arguing that requiring the NSF to circulate peer review reports might "severely damage a merit review system that is the envy of the world." Smith's fellow Texan Eddie Bernice Johnson (D-TX) said Smith was "sending a chilling message to the entire scientific community that peer review may always be trumped by political review." And President Obama himself, in his April 29 address to the National Academy of Sciences on its 150th anniversary, assured listeners that he would work to protect "the integrity of the scientific process" from "political maneuvers." 

I recently began a project on the development of peer review in the twentieth century, so I am always interested when peer review pops up in the news. But the main reason the HQRA debate piqued my interest is that it's almost identical to a controversy about NSF funding from 1975. 

See if this sounds familiar. In 1975, Senator William Proxmire (D-WI) expressed concern about whether NSF spending was benefitting the American public. Proxmire named five NSF-funded grants that he said were "at best, of nominal value to the American taxpayer who foots the bill." He then called NSF director H. Guyford Stever before the Senate to defend the NSF's spending decisions.

Unlike Smith, whose proposals don't seem to have attracted much support, Proxmire found Congressional allies in Rep. John Conlan (R-AZ) and Rep. Robert Bauman (R-MD). Bauman proposed that the NSF should submit all grants for Congressional approval before promising any funding. Conlan, like Proxmire, believed that the NSF's grants were of minimal benefit to most Americans and that they were disproportionately awarded to elite private universities in the Northeast. Conlan quickly became one of the NSF's harshest critics.

Here's where things get interesting. One of the criticisms Conlan lobbed at the NSF was that the organization was ignoring referee opinions when it made funding decisions. In other words, Conlan set Congress forth as the defender of peer review.

Inside the review process (Source: Wikimedia Commons)

Conlan's criticisms appeared to strike a nerve with the NSF. In the early days of the organization, referees were seen as advisors to the division directors, but the directors retained the power to fund projects with lukewarm reports or reject proposals with enthusiastic ones. This was typical of peer review processes in the 1940s and 1950s, which tended to give editors and grant organization employees the power to accept or ignore the referees' advice as they saw fit.

But by the 1970s, the attitude towards peer review in the United States had changed. Peer review was increasingly being linked with scientific legitimacy. (Figuring out how and why this happened is the goal of my new project.) The idea that NSF directors might award a grant to a proposal with lukewarm referee reports was less acceptable to the scientific community—and to the public. NSF officials responded to the 1975 criticisms by placing more responsibility for decision-making on referee reports. A new audit office at the NSF was created to ensure that directors placed appropriate weight on positive and negative reports.

Director Stever then used the peer review reforms to justify rejecting Proxmire, Bauman, and Conlan's other proposals. Stever and other NSF officials argued that having proposals reviewed by experts was the best and only way to decide which projects should be funded. No further scrutiny was needed to guarantee that good science was funded and poor science was not, especially scrutiny from non-expert reviewers such as members of Congress.

The strategy was successful; the suggestion of Congressional review for NSF grants was dropped. Essentially, the NSF and Congress agreed to place their trust in peer review in order to determine how the NSF's chunk of taxpayer dollars would be spent.

http://strange-matter.net/screen_res/nz060.jpg (with permission)

Are the parallels between Smith's ideas and the 1975 proposals just an amusing case of déjà vu? Maybe, but there are some differences that are worth noting. In 1975, the criticisms of NSF spending led to Congress placing more trust in the NSF's peer review process. Thirty-eight years later, Smith seems to think that this trust might have been misplaced. In a May 3 interview with Science, an anonymous Science Committee aide explained that the HQRA was designed to add an extra step in between the referee reports and approval of the grants. As he put it: "There is a step between peer review and the awards being made, and somewhere in there, Congress is saying, 'We think an additional step is needed to solve the problem of so many questionable grants being awarded.'"

In other words, Smith's office seems to think that NSF reviewers can't be trusted to approve good projects and reject inadequate ones. An extra step is needed to make sure nothing "questionable" receives funding.

Does the HQRA signal that people outside the academy are losing their trust in scientific peer review? Actually, I think it's just the opposite. The speed with which the HQRA appears to have died on the vine suggests that public faith in peer review is still quite robust. Notably, in the interview I linked above, the Science Committee aide went out of his way to convince the reporter that the HQRA was not interfering with peer review itself. No one seems to think that attacking peer review is going to be a winning strategy.

In fact, trust in peer review might just be stronger outside the academy than within it at the moment. Scholars in many fields have written volumes on whether peer review actually works the way we want it to. I will be interested to see if the furor surrounding the HQRA dampens these kinds of critiques. When "trust in peer review!" has been such an effective rallying cry for the pro-NSF crowd, will scientists and other scholars want to criticize their best defense against Congressional interference?

Steven Pinker's New Scientism

Yesterday, The New Republic published a big article by bestselling Harvard psychologist Steven Pinker. The title says it all: "Science Is Not Your Enemy." Or does it? After all: whose enemy is science supposed to be? Pinker's answer is there in his subtitle: the targets of his "impassioned plea" are "neglected novelists, embattled professors, and tenure-less historians."

http://upload.wikimedia.org/wikipedia/commons/8/83/Steven_Pinker_2005.jpg

Humanists: according to Pinker, science isn't your enemy—it's your friend. Or your extremely successful younger sibling. Its methods and results are yours if you want them—all you have to do is ask. The problem is: you don't want them—you shy away from science, or reject it outright.

Pinker's got a solution, and he's calling it "scientism."

As Pinker points out, "scientism" is a term of abuse. It's usually hurled at "reductionist" efforts to pose scientific solutions to all sorts of problems. And, as a barb, it's often hitched to times when bad politics wore a scientific mask—Social Darwinism, say, or eugenics. According to Pinker, this is how some people paper over ignorance and fear of the sciences.

By appropriating the term, Pinker hopes to wipe the slate clean (sorry). He sees the new "scientism" as a campaign to both "export" scientific ideals to "the rest of intellectual life" and add scientific ideas to the stock of existing "tools of humanistic scholarship." I'll come back to both this idea of exportability and the metaphor of the toolkit in a bit.

But first: why all the fuss? Pinker's "scientism" is supposed to help solve the widespread (if perhaps unwarranted) sense that "something is wrong" with the humanities. As Pinker points out, "anti-intellectual trends" and "the commercialization of our universities" are part of the problem. But so is "postmodernism"—in a sense, the humanities have made their own bed.

John Brockman—a self-described "cultural impresario" about whom I've written before—shares Pinker's sense of what's wrong. In the preamble to a re-posting of Pinker's piece, Brockman is even more polemical: "the official culture" has "kicked [science] out" and "elite universities" have "nudged science out of the liberal arts undergraduate curriculum." He sees scientific intellectuals—bestselling authors, MacArthur fellows, TED talkers—as a sort of renegade "subculture."

http://upload.wikimedia.org/wikipedia/commons/d/de/John_Brockman_at_DLD.jpg

Does this sound right? It seems to me that, even within the academy, work that spans "the two cultures" is consistently rewarded—most obviously, with prizes and grants. The cutting edge is often that which is most engaged with the sciences. Say what you want about the digital humanities or experimental philosophy—they seem to be doing alright for themselves.

Interestingly, what Pinker points out as quintessentially humanistic modes of inquiry—"close reading" and "thick description"—stemmed from precisely this sort of engagement. Stefan Collini and John Guillory have revealed the roots of "close reading" in interactions between literary critics and scientific psychologists in the 1910s and '20s. And we owe "thick description" to Clifford Geertz and the cross-pollination of anthropological field-work and cultural history in the 1960s and '70s.

It could be that something similar—a new paradigm, even—is emerging from the adoption of digital tools, statistical methods, and fMRI scans by humanists today. Or not. The point is that such engagement is going on, and has a legacy that spans the twentieth-century—on either side of C.P. Snow's "Two Cultures" diagnosis fifty years ago.

But I don't want to rest on rejecting Pinker's premise. Whether or not the humanities are in crisis, lots of people think they are—and many agree with Pinker that the sciences might offer a way out. What I want to highlight is the consequences of imagining this interaction in the terms I noted above: the "export" of ideals or the "toolkit" approach to rapprochement.

This view of intellectual life is a common one, well-illustrated by the title of a recent book by the philosopher Daniel Dennett: Intuition Pumps and Other Tools for Thinking. It's no accident that Dennett is a leading philosopher of evolution: this view of cognition as tool-using is profoundly Darwinian. As a result, it represents, all by itself, the success of a particular scientific "export."

http://upload.wikimedia.org/wikipedia/commons/2/21/Daniel_dennett_Oct2008.JPG

This model of human agents—as embedded bricoleurs doing their best with the cultural resources ("tools") at hand—is something we've argued about on this blog before. And it might well be the correct view. It's certainly a very compelling one. Pinker, Dennett, and many of their peers in cognitive science and human evolution adhere to it.

And so do humanists—or at least historians. Limiting ourselves just to the history of science, let's think over how the human agents at the heart of recent works are characterized. For the most part, I'd argue, they're painted in a light very similar to the Pinker-Dennett-evolutionary model.

It wasn't always this way, though. Time was, there were earnest efforts by historians to cast human actors in Marxist or Freudian—rather than a Darwinian—roles. In the last half-century, however, such accounts have gone the way of the Dodo, leaving us with one that's extremely assimilable to reigning scientific views.

Here's the rub. Pinker might be right about "two cultures" angst. But in adopting the toolkit model, he's also put his finger on a prevailing assumption that ties the two sides together. This might explain both the promise and the peril perceived in the sort of "scientism" he's proposing. Such shared assumptions are essential for bridge-building. But if humanists are uncomfortable with them, then the theory of agency underlying our accounts might merit further scrutiny.