Institutional Review Blog

The Systematic Threat to Academic Freedom

2009-07-04T09:07:00.002-04:00

Lisa Rasmussen kindly alerted me to her essay, "Problems with Minimal-Risk Research Oversight: A Threat to Academic Freedom?" IRB: Ethics & Human Research 31 (May 2009): 11-16. The essay mostly seeks to rebut the AAUP's 2006 report, "Research on Human Subjects: Academic Freedom and the Institutional Review Board." Rasmussen identifies some important shortcomings in that report, and she raises key questions about the relationship between IRBs and academic freedom. But I am unpersuaded by her central arguments.

Before I address them, I should note the repeated disclaimers within the essay. "I will not settle here the fundamental issue of whether a convincing argument exists that IRB review poses a threat to academic freedom," Rasmussen writes. "A longer explanation of the [AAUP report's] failures is beyond the scope of this paper, but a brief outline is possible." I am disappointed by these limits. Rasmussen devotes significant space to matters peripheral to the question of academic freedom, such as her assertion that researchers whose work was approved by a department--rather than a central IRB--would necessarily merit less legal protection, a claim whose weakness she acknowledges in a footnote. Given only six pages, Rasmussen would have done better to focus on the question posed in her title.

Rasmussen's main argument is that the AAUP report "does not demonstrate that IRBs pose a threat to academic freedom." As she notes, such a demonstration would require a definition of academic freedom, something lacking in the AAUP report. So she offers a passage from the AAUP's "1940 Statement of Principles on Academic Freedom and Tenure": "Institutions of higher education are conducted for the common good and not to further the interest of either the individual teacher or the institution as a whole. The common good depends upon the free search for truth and its free exposition." Emphasizing the grounding of this argument in the search for the "common good," Rasmussen then concludes that "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via harm to individuals."

I believe this is a misreading of the 1940 Statement, for it suggests that any policy aimed at safeguarding the common good is consistent with academic freedom. For example, she could have written, "there is a prima facie claim that research can be subjected to assessment regarding whether it threatens to harm the common good via the promotion of communist overthrow of the government," and that therefore a prohibition on the use of Marxist analysis is consistent with academic freedom.

A more relevant definition of academic freedom can be drawn from the AAUP's 1915 Declaration of Principles on Academic Freedom and Academic Tenure":

The liberty of the scholar within the university to set forth his conclusions, be they what they may, is conditioned by their being conclusions gained by a scholar’s method and held in a scholar’s spirit; that is to say, they must be the fruits of competent and patient and sincere inquiry, and they should be set forth with dignity, courtesy, and temperateness of language . . .

It is, however . . . inadmissible that the power of determining when departures from the requirements of the scientific spirit and method have occurred, should be vested in bodies not composed of members of the academic profession. Such bodies necessarily lack full competency to judge of those requirements; their intervention can never be exempt from the suspicion that it is dictated by other motives than zeal for the integrity of science; and it is, in any case, unsuitable to the dignity of a great profession that the initial responsibility for the maintenance of its professional standards should not be in the hands of its own members. It follows that university teachers must be prepared to assume this responsibility for themselves.

As Matthew W. Finkin and Robert C. Post write in their new book, For the Common Good: Principles of American Academic Freedom, freedom of research depends on "a framework of accepted professional norms that distinguish research that contributes to knowledge from research that does not." (54) While these two experts on academic freedom decline to offer a firm opinion on the legitimacy of IRBs, they take the AAUP's concerns far more seriously than does Rasmussen (69).

The question, then, is whether IRBs, like the boards of trustees that concerned the authors of the 1915 statement, "lack full competency to judge of [scholarly] requirements." Rasmussen suggests that IRBs merely maintain scholarly standards: "The source of the threat to academic freedom via oversight by one’s colleagues is far from clear," she writes, "especially since researchers undergo peer review for research funding and when submitting their manuscripts for publication." But IRB review is not peer review, since it is conducted mostly by people ignorant of the scholarly methods they are reviewing. (See "Why IRBs Are Not Peer Review," and other posts tagged "peer review.")

To make this a bit more concrete, we can examine the exemplary "horror stories" included in the 2006 AAUP report. Rasmussen rejects these as "unelaborated anecdotes with no documenting citations," rather than examining their implications for academic freedom.

Here's one: "A Caucasian PhD student, seeking to study career expectations in relation to ethnicity, was told by the IRB that African American PhD students could not be interviewed because it might be traumatic for them to be interviewed by the student." Or another: "A campus IRB attempted to deny an MA student her diploma because she did not obtain IRB approval for calling newspaper executives to ask for copies of printed material generally available to the public." No peer review process would impose such conditions. If these are not infringements of academic freedom, then nothing is.

Rasmussen is quite right that we should not equate "inconvenience and hassle with abridgement of academic freedom." Yet nor should we dismiss the abridgement of academic freedom as mere inconvenience and hassle. When IRBs impose conditions on research that prevent researchers from conducting the basic tasks of scholarship--talking to people of varied backgrounds, recording interviews, or telephoning for information--they abridge academic freedom. The more interesting questions are how often this occurs, and why it happens.

Rasmussen presents IRB abuse as a somewhat random process: "IRBs can function well or poorly, and which is true for a given IRB depends on many factors, not least of which are institutional support and member training." This suggests that IRB abuses are individual anomalies, rather than a pattern.

By contrast, the AAUP detects a systematic bias toward the infringement of freedom. This is better developed in the AAUP's 2000 report (cited by Rasmussen), "Institutional Review Boards and Social Science Research." That report includes such observations as "no one is likely to get into trouble for insisting that a research proposal is not exempt" and "no university is likely to want to explain to either the government or the public why its commitment to avoid harming the human subjects of research is limited by the source of funding for the research." In these and other cases, the AAUP recognizes that the IRB system punishes individuals and institutions only for approving research, not for restricting it.

The design flaws in the system have yielded a pattern of abuse. Read Maureen Fitzgerald and Laura Stark, both of whom observed repeated abuses by the IRBs they studied. Read Linda Thornton, whose work was thwarted at 15 of 24 institutions she contacted. Read Jack Katz, who shows that IRBs are particularly likely to pounce on controversial topics. IRBs can function well or poorly, but the system is weighted toward poor function.

Rasmussen acknowledges that poorly designed systems can lead to systematic problems. She concedes that the "lack of an [IRB] appeals process may threaten academic freedom." She also details the way that departmental-level review might systematically hamper research. And she ends her essay with a promising proposal for “template review:"

Disciplines at the national level might formulate templates to guide very common research approaches. For example, a research template for oral historians could stipulate that the researcher will interview individuals, record their answers, refer them to counselors if the questions have provoked strong emotions, procure consent forms, lock the transcripts securely, and identify what will happen to the transcripts at the close of research. IRBs at individual institutions would review the template once and approve it (or even decide to accept any templates from given professional societies). Thus, a researcher would simply submit a form to the IRB stating her agreement to abide by the format of the template. Upon receipt of the form, the IRB would approve the protocol.

If IRBs are not threatening academic freedom, why propose this reform? Inside this proposal is an acknowledgment that disciplinary experts and professional societies in the social sciences and humanities have been excluded from the present IRB system. While such exclusion does not automatically threaten academic freedom, we should not be surprised when it does. For all her skepticism of the AAUP report, Rasmussen has presented her own suggestion that the current system is rotten at the core.

Finnish Group Warns Against Unnecessary Bureaucracy

2009-06-29T09:08:00.000-04:00

Klaus Mäkelä and Kerstin Stenius kindly alerted me to their paper, "A New Finnish Proposal for Ethical Review in the Humanities and Social Sciences," which they presented in London in April. The paper describes a draft report by a working group of Finland's National Advisory Board on Research Ethics, which examined the need for ethics review in the humanities and social sciences.

In its draft report, issued in January, the working group adopted some principles that would be familiar to ethics committees and regulators in the United States and other countries. The report stresses the importance of voluntary participation, informed consent, the confidentiality of information, the avoidance of "undue risk and harm," and the need for special care when researching minors. It sees ethics review committees as part of a process to effect these goals.

On the other hand, the working group recognizes that too much oversight presents its own problems:

5. It is important to respect the autonomy and good sense of research subjects. In social research, participants usually are fully competent to assess the risks involved without outside expertise. Ethics committees should avoid paternalism.

8. Clear criteria should be formulated for what kinds of projects require ethical review, but it should be up to the researcher to determine whether a project meets these criteria.

10. The work of ethics committees should be as transparent and open as possible and a system of appeals should be put in place.

The second part of principle number 8 is particularly significant. U.S. regulators have, since 1995, insisted that researchers cannot be trusted to determine when their research is subject to review under the Common Rule. Recently, Jerry Menikoff of OHRP noted that institutions are not legally required to strip researchers of the power to make these determinations, but OHRP will continue to recommend that they do so.

The Finnish working group, by contrast, sees a greater danger in giving that power to committee members and staffers who will likely err on the side of too much review:

It is a matter of judgement to decide what kinds of stimuli are 'exceptionally strong'. To avoid unnecessary bureaucracy, it nevertheless should be up to individual researchers to decide whether their project falls into the categories listed above and needs to be submitted to ethical review. It is highly unlikely that this will lead to transgressions, and ex post facto sanctions will be enough to keep any exceptions under control.

In short, the working group understands that in this case, the dangers of too much bureaucracy outweigh the dangers of too little.

I should note that the London conference at which Mäkelä and Stenius presented their work was the Third Working Meeting of the International Study of Ethical Codes and Ethical Control in the Social Sciences, the previous conferences having been held in London in 2007 and 2008. The meetings have brought together scholars from several northern European countries to discuss social science ethics and regulations across international borders. It is splendid that these scholars are at work on so important a topic, and I look forward to learning more from them.

Menikoff to Critics: "Yes, We Hear You"

2009-06-17T08:54:00.003-04:00

Theresa Defino kindly alerted me to the streaming video feed of Dr. Jerry Menikoff’s May 14 address at the University of Michigan, “The Legal Assault on the Common Rule." The speech was an impressive acknowledgment of the widespread criticism of the foundations of the IRB system, and it ended with the promise of some substantive improvement. But by listing some of the most common critiques of the IRB system without attempting to rebut them, Menikoff fell short of the dialogue he seeks to foster.

Menikoff began his speech with a word for IRB critics: "Yes, we hear you." He then spent most of his hour summarizing some of the more prominent critiques of the U.S. IRB system. Among them:

Carl Schneider, Personal Statement, in President's Council on Bioethics, The Changing Moral Focus of Newborn Screening: An Ethical Analysis by the President's Council on Bioethics, 2008.

Department of Health and Human Services. Office of Inspector General, Institutional Review Boards: A Time for Reform (1998)

Philip Hamburger, "The New Censorship: Institutional Review Boards" (2005).
University of Illinois, The Illinois White Paper. Improving the System for Protecting Human Subjects:. Counteracting IRB “Mission Creep” (2005).

Northwestern University Law Review, Symposium on Censorship and Institutional Review Boards (2006)

American Association of University Professors, Research on Human Subjects: Academic Freedom and the Institutional Review Board (2006)

(As Menikoff noted, challenges to the legality of the IRB regime are only one component of this broad critique, and Menikoff declined to go into detail about them. This makes the title of his speech an odd one.)

For most of the speech, Menikoff was carefully respectful of the authors of these documents, noting their prominent positions and accomplishments in other areas of scholarship. The only real rebuttal he made was to suggest that some critics are too quick to assume that an IRB would demand to review, or even deny, a particular study. That's a good point; it's always better to critique the system with a real horror story about thwarted research, rather than a hypothetical one. What Menikoff didn't say is that there are plenty of real horror stories to go around.

Menikoff offered one important concession. In his 2007 article, "Where’s the Law? Uncovering The Truth About IRBs and Censorship," he had suggested that if an institution required IRB review of research exempted by the Common Rule, researchers at that institution shouldn't blame the feds:

An institution can choose to impose rules that are more restrictive than [federal] regulations. But complaints about censorship resulting from such a circumstance would seem more appropriately directed at the specific institutions that are choosing to do this, rather than the IRB system itself, as created by the federal regulations. (792, n. 6)

Such analysis ignored the role of OPRR/OHRP in encouraging institutions to impose requirements beyond those in the regulations. In particular, in 1995, OPRR advised institutions "that investigators should not have the authority to make an independent determination that research involving human subjects is exempt." More recently, OHRP contributed to a 2008 report that recommended expedited review for several hypothetical projects that would seem to merit exemption.

In his Michigan speech, Menikoff did better. He conceded that OHRP continues to recommend that researchers should not make exemption determinations. But he also noted, "it's just a recommendation. You don't have to follow it." He pledged that OHRP would clarify this.

While this concession is welcome, it is pretty small stuff. Even on the narrow issue of the Common Rule exemptions, it ignores the question of why OHRP continues to promulgate guidance that contradicts the intent of the authors of the exemptions and that originally emerged from the panic of the mid-1990s that almost everyone agrees led to the overregulation of human subjects research. Nor can I put much faith in a promise that new guidance from OHRP is just around the corner.

More importantly, it's frustrating that Menikoff missed the opportunity to address the central problem posed by the critics he cited: there is no evidence that the IRB system does more good than harm. The question of self-exemption is important, but in the context of the overall critique of the system outlined in the speech, Menikoff's emphasis on this one point becomes a bit of a red herring.

Finally, I must object to Menikoff's patronizing remarks about the emotional state of IRB critics:

It's not as if everything or even the bulk of what these people are claiming is necessarily true or valid, but nonetheless we again have to be aware of the strength of their feelings and the source of it. There is this heartfelt feeling about at least parts of the human protection system that they are wrong.

Yes, critics are angry, but it is insulting to portray us as so overcome with emotion that we cannot form "true or valid" complaints. It is not the strength of our feelings that should concern Menikoff, but the accuracy of our facts and the logic of our arguments.

Lisa Wynn's Words and Pictures

2009-06-05T16:26:00.000-04:00

In April I commented on the ethics training program for ethnographers developed by Lisa Wynn of Macquarie University with some colleagues.

At Culture Matters, the blog of the Macquarie anthropology department, Wynn described the ideas that led her to develop the program.

Now, at Material World, a blog hosted by the the anthropology departments of University College London and New York University, Wynn describes another aspect of the training program: the pictures.

Wynn explains that along with its medical-centered ethics and jargon-laden text, the standard NIH ethics training program suffers from clip art in which people are depicted as faceless cartoons--probably not the best way to get researchers thinking about others as autonomous individuals. So for her program, Wynn offers pictures of real researchers and research participants, from Laud Humphreys to Afghan school administrators.

Gathering these photos--about a hundred in all--wasn't easy, but they contribute meaningfully to the warmth and depth of the site. And it put Wynn in touch with some prominent scholars.

[Side note: Professor John Stilgoe tells his students that it's rare to have enough photos of yourself at work. That's a good admonition; you never know when someone will want to show you doing controversial research.]

In another posting on Culture Matters, Wynn describes her continuing research on research ethics. She notes that ethics-committee oversight of ethnography is a relatively recent phenomenon. While it was debated as early as the mid-1960s, only in the 1990s did it become widespread. Thus, in studying the effect of ethics committees,

We’ve got a perfect “natural” control: an older generation of researchers who spent most of their careers not seeking ethics clearance, a younger generation for whom it is standard operating procedure, and a “middle-aged” group of researchers like myself who started their research under one regime and now live under another (I swear, this is the first time I’ve thought of myself as middle-aged). By correlating responses with different regulatory regimes, we can ask questions like: do researchers who never got ethics clearance have different ideas about what is ethical than researchers who go through ethics review? Does one group consider itself more or less ethical than the other? Or do they feel like ethics oversight hasn’t made any difference to their research practice?

Wynn plans to contact scholars in Australia and the United States to see how the spread of ethics review affected ideas about research ethics. I'm quite excited by this work; in fact, I plan to publish it in a special issue of the Journal of Policy History I am editing on the general topic of the history of human research ethics regulation.

How many pictures should I demand?

A Horror Anthology

2009-05-13T22:28:00.002-04:00

Mark Kleiman takes on IRBs at The Reality-Based Community. On April 14 he asked his readers for IRB horror stories, and on May 2 he posted some of the responses.

The saddest concerns a group of law students who wished "to send testers of different races in different styles of clothing to the restaurant over some period of time to test whether they enforced their dress code in a discriminatory manner." Law school administrators told them they would have to secure IRB approval. This discouraged the students, who did not want to go through the time and effort of the approval process.

This was not the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. At its 15 April 1978 meeting, the commission discussed just such a scenario (pp. II-5 to II-21 of the transcript), and all the members seemed to agree that such testing for discrimination should not require IRB review. But, as I've noted before, the commission wrote a definition of human subjects research that plausibly includes a great deal of activity the commission did not seek to regulate. Thirty years later, justice suffers as a result of the commission's sloppiness.

NOTE: In honor of Professor Kleiman's search, I have gone back through this blog to add the "horror stories" tag to some posts that should have had it to begin with. Clicking on that tag now yields more than a dozen posts, with even more documented horror stories.

Journal of Policy History

2009-05-08T21:43:00.001-04:00

The Journal of Policy History has published my article, "How Talking Became Human Subjects Research: The Federal Regulation of the Social Sciences, 1965–1991." As permitted by the transfer of copyright, I have posted a PDF on my personal website: http://www.schrag.info/research/howtalking.html.

Not much has changed since I posted a version on SSRN in April 2008. The major changes come in the "medical origins" section; the earlier draft underestimated the strength of social scientists' opposition to IRB rules in the late 1960s. Also, the new version better explains the origins of Ithiel de Sola Pool's concern about IRBs (see p. 18).

DeGette Still Doesn't Get It

2009-05-05T08:35:00.002-04:00

Representative Diana DeGette (D-CO) has introduced the Protection for Participants in Research Act (H.R. 1715), which would impose IRB requirements on all human subject research supported by the federal government or affecting interstate commerce.

As amp&rsand notes, this is the sixth time DeGette has introduced this bill. And I don't think that counts earlier submissions of similar bills by Senator John Glenn. None of these previous efforts went far, so there's no particular reason to fear this bill's passage.

Still, it is disappointing that DeGette has introduced this bill six times without understanding its potential consequences. Her press release states that “I think one thing we can all agree on in a bipartisan way is that we need to encourage medical experimentation but we need to do it in a way that both protects the patients and gives them informed consent about what they are getting into," as if the bill would affect only medical experimentation. It points to medical trials in 1999 and 2006 as evidence of insufficient oversight, and argues that "research is the key to innovation and discovery, including curing deadly disease." Nowhere in the press release is a hint that DeGette understands that her bill would outlaw most journalism, not to mention further inhibiting social science and humanities research.

Thirty-five years after the passage of the National Research Act, Congress still doesn't know what it has done.

Bonus question for Rebecca Tushnet's 43(B)log: Is the copyright claim on DeGette's press release--the work of a federal employee in her official capacity--illegal, or merely false?

Update 8 May 2009: A correspondent notes that the copyright statement is no longer on the site. If DeGette removed the statement in response to this blog, good for her. I have a PDF of the press release as it appeared on May 4, if anyone is interested.

UMKC's Respectful Oral History Policy

2009-04-25T23:26:00.002-04:00

The University of Missouri-Kansas City (UMKC) has posted a promising new policy: "Social Sciences IRB and Oral History."

The policy has a number of elements that set it apart from the typical university policy, which seeks to cram oral history into a system designed for medical experimentation. Instead, it adapts only those elements of the medical IRB system that encourage historians to follow their own discipline's ethics and best practices.

I suggest that readers of this blog read the whole policy, but here are some highlights:

1. Respect for Critical Inquiry

As I have written repeatedly on this blog, historians do not take the Hippocratic Oath, and should not promise not to harm the people they interview. Any IRB that imposes the Belmont Report on historians is asking them to forswear their own ethics.

UMKC understands this. Its policy notes that

akin to a journalist or lawyer, an historian is also responsible to a wider public to recover a shared past “as it really happened.” In keeping with the public role of an historian in a democratic society, these responsibilities, especially when conducting narrative interviews, can necessitate a confrontational style of critical inquiry. So while historians do not set out to hurt their interviewees, oral historians are expected to ask tough questions in their interrogation of the past.

2. Respect for Peer Review

The UMKC neither subjects oral historians to the whims of board members unfamiliar with their field, nor does it leave them on their own. Instead, it offers scholars a number of relevant readings, including publications of the Oral History Association, and then encourages them to talk to colleagues knowledgeable about interviewing:

After reviewing these resources on their own, the researcher is strongly encouraged to discuss their research protocol with peers before implementing their research protocol. In some cases, peer review by members of one’s own department would be most useful; in other cases, a researcher might be better served by seeking review from a colleague in a different department.

To foster these kinds of conversations among the faculty, the Social Sciences IRB Subcommittee for Oral History will hold two meetings per semester . . . to discuss “Best Practices” in oral history. Faculty experts in oral history will guide these conversations . . . These meetings are designed to meet the needs of researchers seeking advice and peer review for their research protocols. They are also designed to meet the needs of Chairs and/or designees interested in learning how to advise researchers in their departments to make responsible decisions regarding oral history.

3. Respect for OHRP's Pledge

UMKC takes seriously the carefully negotiated 2003 agreement between the American Historical Association and the Oral History Association and OHRP, even posting a copy on its website. The university elaborates on that agreement:

At UMKC, we draw a distinction between idiographic research that uses oral histories to describe the unique story of some particular social group or individual, which does not constitute “human subjects research”; and nomothetic research that employs oral histories in the hopes of contributing to a general theoretical or comparative debate about the human nature or behavior, which does fall under the category of “human subjects research."

While I confess that the terms idiographic and nomothetic are not in my working vocabulary, I believe they do express a real difference between the ethics of oral historians and those of other scholars. If one is interested in a general theoretical or comparative debate about the human nature or behavior--as many social scientists seem to be--then it makes less sense to single out individuals for potential honor or calumny. Writing about unique individuals or groups changes one's responsibility toward the individuals interviewed.

4. Respect for Researchers

Policies like UCLA's infantilize researchers, making them submit every judgment to an administrator. By contrast, UMKC trusts its scholars:

The bottom line is that the researcher makes these determinations in careful consultation with the Chair of the department or another official designee appropriate to the kind of study being planned. Together this determination is based on shared understanding of all relevant guidelines and their shared expertise in their specialized field of scholarship.

5. Respect for the IRB

Even as it empowers historians, the UMKC policy keeps the IRB involved, making it a resource, rather than an obstacle. Researchers still have to learn something about human subjects regulations, and they must complete a form explaining why they have determined that their policy does not fall under federal regulations.

(The form's demand for an explanation of "no more than 1500 characters" sounds suspiciously bureaucratic, but it's a good length for the presentation of a single idea--about the same as the 150-word limit for a New York Times letter to the editor.)

More importantly, the frequent meetings of the Social Sciences IRB Subcommittee on Oral History suggest that some scholars at UMKC have devoted their time to helping colleagues deal with the real ethical challenges of oral history.

The website explaining the policy notes that it was developed by "a group of faculty and administrators involved with the Social Science Institutional Review Board (SSIRB) . . . with input from members of the SSIRB, the College of Arts & Sciences, and the Faculty Senate at UMKC." I congratulate all the scholars and administrators who developed this innovative system, and I hope it works as well in practice as it reads on the screen.

With this policy, UMKC joins Amherst College, Columbia University, the University of Michigan-Ann Arbor, and the University of Nebraska-Lincoln Policy on a small but growing list of schools that have adopted OHRP's 2003 position removing most oral history research from IRB jurisdiction. Five schools not very many, but it's five more than the AHA could find in February 2006. Who will be number six?

Deregulation "Is Not Going to Happen"

2009-04-21T22:01:00.003-04:00

Linda Shopes kindly alerts me to the April 20 issue of COSSA Washington Update, the newsletter of the Consortium of Social Science Organizations, which reports on an April 1 meeting of the National Academies’ Board on Behavioral, Cognitive, and Sensory Sciences, at which IRBs were discussed.

Here's the key passage:

Philip Rubin, CEO of Haskins Laboratories in New Haven, CT, and former director of the National Science Foundation’s (NSF) Division of Behavioral and Cognitive Sciences, chairs the Board. He began the session with a review highlighting the difficulties social/behavioral researchers have had with the current system under the Common Rule regulation and its interpretation by campus Institutional Review Boards (IRBs). Complaints have been loud, but mostly anecdotal . . . Once again the bottom line is that despite efforts by Joan Sieber and the Journal of Empirical Research on Human Ethics, which she edits, there are still large gaps in our empirical knowledge of how the system works for social and behavioral scientists.

Rubin was followed by Jerry Menikoff, new head of the U.S. government’s Office of Human Research Protections (OHRP). Menikoff announced that he was all for “flexibility” in the system and that “changes can be made.” He also endorsed conducting more research. He rejected the arguments of the American Association of University Professors and Philip Hamburger of Northwestern University Law School that IRBs violate researchers’ first amendment rights. He acknowledged the importance of expedited review, but stated quite clearly that “removing minimal risk research from the system is not going to happen.”

I don't want to make too much of these comments; an OHRP spokesperson tells me that they were an extemporaneous response to Rubin, and not prepared remarks. Still, I am disappointed. Menikoff's comments suggest a retreat from his earlier concession that "flexibility" often can be code for arbitrary power. And it's a pity for a public official to insist that a given policy "is not going to happen" even as he endorses more research. Wise governance depends on making policies after finding facts, not before.

Macquarie's Innovative Ethics Training

2009-04-17T06:36:00.005-04:00

In previous posts and my 2007 essay, "Ethical Training for Oral Historians," I have complained about standardized, medicine-centric ethics training systems like the CITI Program and called for training programs better tailored to individual disciplines.

Lisa Wynn of Macquarie University (also known as MQ) has alerted me to just such a program she created with Paul H. Mason and Kristina Everett. The online module, Human Research Ethics for the Social Sciences and Humanities, has some elements that I find inappropriate. Overall, however, it is vastly superior to the CITI Program and comparable ethics programs I have seen, and it deserves attention and emulation.

Strengths

Relevant Examples

The CITI Program's "History and Ethics" module offers five "Events in Social & Behavioral Research" as cautionary tales. Two concern psychological experiments (Milgram and Zimbardo), two concern decades-old observations using deliberate deception (Wichita Jury and Laud Humphreys) and the last is Francis Flynn's more recent, but also deliberately deceptive, study of restaurant responses to customer complaints. This choice of examples suggests that the only ethical dilemmas facing ethnographers concern deception, and that they do not occur frequently. The University of Iowa's Ethical and Regulatory Issues in Ethnographic Human Subjects Research is broader in its concerns, but offers few concrete examples.

The MQ program, by contrast, offers two old stories (Zimbardo and Humphreys), but it freshens them with stories of two recent controversies: the U.S. Army's recruitment of social scientists for its Human Terrain System initiative and Sudhir Venkatesh's doctoral research in underground economies in Chicago. Another case study appears in the section on "Research with Aboriginal and Torres Strait Islander Peoples," describing the collection of stories, songs, and artifacts by anthropologist Ted Strehlow. These collections have become the subject of considerable controversy, and the program offers both Strehlow's perspective and that of his critics. Beyond these case studies, the MQ program offers short vignettes in which ethnographers had to decide how to conduct research without hurting their informants or themselves. In some cases, the researchers are given pseudonyms; in others, real names. The authors even include accounts of dilemmas they themselves have faced.

The stories that make up the bulk of the MQ program are more relevant to today's ethnographers than hearing about what Laud Humphreys did forty years ago, or about recent medical research gone wrong. They show that ethical challenges are not confined to the distant past, but face social scientists today. And they broaden the challenges facing scholars to questions of confidentiality, government sponsorship, intellectual property, and other contemporary concerns.

Best of all, these stories emphasize real ethics, rather than the regulatory compliance at the heart of the typical program. When I went back to the CITI Program to write this entry, I was immediately confronted with a quiz question about whether a hypothetical foreign institution needed its own federalwide assurance for a hypothetical project. I'm sure this is very important to some IRB administrators, but it is absurd to demand that every researcher master such arcane requirements. The MQ program is not wholly free of this dross, but it is better than others I have seen.

Like other training programs, the MQ program is heavy on examples of what not to do. But it also offers something else: a positive example. One section describes how medical anthropologist Paul Farmer not only studied tuberculosis in Haiti, but also established a medical clinic there. As inspirational as this story is, I think this section could be expanded to offer stories of ethnographers who helped communities simply by doing good ethnography--not by providing medical care. Oral historians often get to work with people who are happy to have their words recorded for posterity. I would hope that sociologists and anthropologists have equally gratifying experiences. The MQ program does offer additional readings, which may contain these kinds of examples.

Room for Debate

The CITI Program lists "Events in Social & Behavioral Research" with attendant "ethical problems." The MQ program, by contrast, describes "ethics controversies" and "debates about the ethics," suggesting that there is no single right answer to the conundrums faced by scholars in the field.

To emphasize that point, the MQ program rejects the single-answer, multiple-choice quizzes that are the staple of the CITI Program. Instead, the MQ program offers questions like this:

Say you are doing research on cigarette smoking, but as you talk to the smokers, they start telling you about the illicit drugs they use. What do you do? (47)

There are no multiple choices here, just a stark presentation of an ethical challenge a scholar might reasonably face.

Elsewhere, the program does offer multiple choice quizzes, but not with the simplistic approach of the CITI Program. For example, one screen invites the user to choose one of four strategies for avoiding the revelation of confidential information. Whatever one chooses, the program replies that "any of the above are possible strategies for protecting your informants' identities, but some are better strategies than others," and then elaborates.

The program even admits that university ethics committees don't have all the answers, and sometimes have the wrong answers. It offers the example of Kristina Everett, one of the program's authors, who offended a longtime friend by confronting her with the written consent form demanded by her university. The program concludes,

Institutional ethics rules can . . . fall short of researcher’s own moral responsibilities and commitments to a particular cause. Ultimately, researchers must make their own decisions about what is ethical in the context of the particular research situation, in dialogue with their research participants. (103)

Scholarly Norms

The MQ program respects its students by adhering to two basic norms of scholarly writing.

The first is citation. Many of the CITI Program's statements are unattributed, and in some cases factually inaccurate. When a statement is attributed, the program often gives just a single source. The MQ program, by contrast, provides several readings for each of the case studies it presents, in some cases with alternative viewpoints.

The program also provides citations for statements throughout the text, adding authority. A warning that the U.S. government might seize a scholar's laptop at an airport might sound alarmist were it not for a link to a news article on the subject.

Another norm is that of open access. Many ethics training programs are open only to affiliates of a single university or, in the case of CITI, to affiliates of institutions that subscribe to the service. This robs scholars of the chance to compare and critique rival systems, to find areas of agreement and disagreement, to check facts, and to do all the other work that scholars usually do in their quest for truth and wisdom.

The MQ program requires registration, but that registration is free and open to all. Better still, the authors have licensed the text under a Creative Commons Attribution Non-Commercial Share Alike license, giving others the chance to "download, redistribute, remix, tweak, and build upon the text of this work non-commercially, as long as they credit the original authors and license their new creations under the identical terms." These two decisions offer the opportunity and the challenge for other institutions and disciplines to provide even better training.

The downside of such openness is that it leaves a training program open to criticism from cranky bloggers halfway around the planet. Let's move on to some of the program's weaknesses.

Areas for Improvement

Irrelevant History

The first section of the Macquarie program takes readers through about 15 screens concerning the history of abuses by medical researchers--Nazis, Tuskegee, and the more recent, dubious drug trials in Africa--along with descriptions of the various ethical codes designed to prevent recurrences. I hate this. It presents biomedicine as the archetype of research and social research as a deviation from that norm. And it's just bad pedagogy to start a course with material not important to the student.

If this is truly training for the social science and humanities, why not tell just the story of ethical debates in the social sciences and humanities? Or, if social scientists must know about Nuremburg and Tuskegee in order to communicate with university ethics officers, why not leave that to the end?

Disciplinary Bias

The MQ program is entitled "Human Research Ethics for the Social Sciences and Humanities" and it notes that "social science research -- including psychology experiments, quantitative surveys, oral history collection, and ethnographic research or participant observation -- raises its own peculiar problems for ethical research practice." (24) But the program does not give equal weight to the problems of each type of research or the ethics embraced by its practitioners.

The case studies include no examples of psychological experimentation since Zimbardo's prison study, or any survey or oral history research, or other fields not included in the list, such as geography, journalism, and creative writing. (Keep in mind that the Australian National Statement on Ethical
Conduct in Human Research concedes that its broad definitions "could count poetry, painting and performing arts as research." [2:8]) Instead, almost all of its examples, and its quotations from ethics statements, come from anthropology.

Presenting case studies and ethical statements from all the fields in the social sciences and humanities would bloat the training program beyond usefulness. Rather than add more content, the program should strip references to survey and oral history research and adopt a more humble title: "Human Research Ethics for Ethnography."

Even this might be problematic, given the divergence in research ethics among various stripes of ethnographers. In particular, the program ignores debates over the principle of beneficence. In describing the controversy over the Pentagon's Human Terrain System (HTS), the authors write:

The reason for anthropological opposition to the Human Terrain System (HTS) lies in the discipline's code of ethics. Specifically, HTS opponents charge that Human Terrain Teams cannot guarantee several basic tenets of ethical research, including 'Do no harm,' the principle of informed consent, and freedom from coercion. (40)

Since when are these "basic tenets" of ethical research? For anthropology, the answer would seem to be only since the 1990s. [Carolyn Fluehr-Lobban, "Ethics and Professionalism in Anthropology: Tensions Between Its Academic and Applied Branches," Business and Professional Ethics 10 (1991): 57-68.] For other scholarly disciplines, the answer is not yet, particularly since the idea of "do no harm" would preclude investigative journalism and other critical inquiry. For many scholars, bringing harm to a malfeasor is highly ethical practice, but the MQ program seems to think that all scholars have embraced anthropology's gentleness.

I find this narrowness particularly disappointing because just last summer, Wynn herself raised the question of disciplinary differences in a post about HTS on Culture Matters. She understands that anthropologists are unusual in some of their ethical beliefs, and I wish that insight had made it into the MQ training.

(See "My Problem with Anthropologists.")

Incomplete Accounts

I have additional concerns about the treatment of two of the four social-science case studies in the program.

The section on Laud Humphreys is somewhat ahistorical, neglecting the ethics debates that were raging through sociology at the time Humphreys planned his work. I'm also perplexed by the statement that "Humphreys argued that the deception was justified, because there was no other way to obtain such information in a consented way because such sex practices were so highly stigmatized." (37)

Really? Here's what he wrote in 1975: "I am forced to agree with my critics regarding that part of my study in which I traced license numbers and interviewed respondents in their homes . . . I now think my reasoning was faulty and that my respondents were placed in greater danger than seemed plausible at the time.” [Laud Humphreys, Tearoom Trade: Impersonal Sex in Public Places, enlarged ed. (Chicago: Aldine, 1975), 230.]

As for the section on the Human Terrain System, it is based almost entirely on the writings of critics. For example, the program asks,"If the anthropologist has no control over what the military will do with the information, then how can an anthropologist fully explain to people the real risks involved in research?" Not only does this question ignore the fact that not all Human Terrain team members are anthropologists, it also assumes that those members do not control the information they gather. For a challenge to this assumption, see Adam Silverman, "The Why and How of Human Terrain Teams," Inside Higher Ed, 19 February 2009.

Awkward Layout

Writing this review was slowed considerably by the poor design of the MQ program's site. The program consists of 103 screens which must be viewed one at a time. Since one must often scroll down to reach the "next" link, it took hundreds of mouse actions to get through the site. With a big enough monitor, you can enlarge the window to include all the text and the navigation links without scrolling, but then you end up with line lengths of 100 or more characters, making the text hard to read.

If you stop clicking for a while, the site logs you out. And while you can return to the chapter you were on, you cannot navigate straight to the page. So if you pause to take notes, as I have while writing this entry, you may need several hundred mouse clicks to get through the whole program. I also found a noticeable pause after each screen, further delaying my reading. There is no way to print or download the site's text in large chunks.

Text this important shouldn't be an ordeal to read, and I am sure than the authors would never try to publish a journal article of comparable length in this format. They need to find a better way to present this material. For example, they could collapse the 100+ screens into just six pages--one for each of the six major sections--allowing users to view on screen, print, or save as needed.

Trivia Quiz

The MQ program ends with a quiz composed of a few dozen multiple-choice questions about the material covered in the main text. In several cases, I found none of the answers satisfactory. In other cases, the quiz presented two opinions as choices and demanded the "right" one. More frequently, the quiz posed silly questions, like asking about what provisions are included in which codes. Much as I believe in the power of historical knowledge, knowing whether the Nuremberg code included the right to withdraw from an experiment is of no obvious use to an ethnographer working today. And the quiz didn't even tell me which questions, if any, I got wrong. The quiz section was also heavy with typographical errors, making me wonder if it was written by someone other than the main authors of the MQ program. It is a shame to end such an innovative program on so sour a note.

There may be a bureaucratic need to include a quiz at the end; some administrator at MQ can now check that I have completed the training. But I see little pedagogic need for the quiz, since the program itself (after the medical ethics portion) is bound to be compelling to any ethnographer concerned with ethical research.

Ethical Education

Wynn, Mason, and Everett have put tremendous thought and effort into determining what ethnographers need to know about research ethics, and packaging that knowledge in a way that is respectful of researcher's learning, intelligence, curiosity, and desire to act ethically. They go too far when they claim that the ethics currently embraced by Australian and American anthropologists are universal to social scientists, and they at times make statements that lack nuance. But rather than proclaim these ideas as gospel, they have designed a training program that itself invites debate. I hope that other universities and other scholarly disciplines will follow their example.

Training Day

2009-04-10T13:38:00.000-04:00

Peter Klein of the Organization and Markets blog offers a sad account of what it takes for a University of Missouri economist to gain permission to interview entrepreneurs or hand out surveys to corporate executives. Like many scholars across the country, he was directed to an online training system, which demanded that he provide correct answers to questions like the following:

32. The investigator is a 1/8th V.A. employee. She proposes to recruit MU outpatients into a study conducted exclusively at MU facilities. Which of the following groups must approve the research project before participants can be enrolled?

* The MU Health Sciences Center IRB
* The V.A. Research and Development Committee
* Both a. and b.
* Neither a. nor b.

While such knowledge may be of critical importance to health researchers at Missouri, it is irrelevant to social scientists not doing medical work. The lesson Klein takes away from such an experience is not that he must be sure to obey laws and ethics standards while doing his research, but that his campus IRB administrators do not respect him enough to provide relevant ethical training.

Administrators take note: you are making fools of yourselves, and earning your faculty's contempt.

See Comments Oppose New Regulations on Training.

Deadline Extended for TCPS Comments

2009-03-29T13:56:00.003-04:00

John Lowman kindly alerts me that Canada's Interagency Advisory Panel on Research Ethics has extended the deadline for comments on the draft second edition of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS). Comments will now be accepted through 30 June 2009, though the PRE encourages comments by March 31, since the next round of revision will begin in April.

An official announcement of the deadline extension is online at the PRE's French-language website. I could not find an English-language version on the PRE website, but the University of Western Ontario has posted one.

A form for online comments, and instructions for submitting comments by mail, fax, or e-mail, is online.

I have sent in a version of the comments posted on this blog. As I prefaced my comments to the PRE, I write as a non-Canadian. But the regulation of research ethics is an international endeavor. Just as TCPS draws heavily from the Belmont Report and 45 CFR 46, so can we expect TCPS to influence American policy and guidance. I therefore consider myself to have some stake in the outcome of the TCPS revision.

Canadian Criminologists Decry TCPS Draft

2009-03-18T09:17:00.000-04:00

Back in January, I mentioned the release of the Draft 2nd Edition of the Tri‐Council Policy Statement: Ethical Conduct for Research Involving Humans, prepared by Canada's Interagency Advisory Panel on Research Ethics, or PRE.

Ted Palys and John Lowman of the School of Criminology, Simon Fraser University, kindly alerted me to their critique of the draft, or TCPS-2, as they term it. (They even more kindly cited this blog in their work.) They find that TCPS-2 "poses a significant threat to academic freedom in Canada." (3)

Their 20-page critique, "One Step Forward, Two Steps Back: Draft TCPS-2’s Assault on Academic Freedom," is all meat and no fat, and I recommend that it be read in its entirety. But here are a few salient points.

1. TCPS-2 Tells Researchers Not to Fight Subpoenas

TCPS 2 states that "researchers may face situations where they experience a tension between the requirements of law and the guidance of ethical principles. In such situations, researchers should do their best to uphold ethical principles while complying with the law." (p. 17, lines 598-602).

Palys and Lowman note that such advice yields what they consider to be the wrong answer to the only recent incident involving a Canadian social researcher who came into conflict with the courts: Russell Ogden's studies of assisted suicide. Palys and Lowman have written about that case at length, and they applaud Ogden's refusal to testify about information given to him after he promised confidentiality.

While I understand the authors' wish that TCPS-2 include room for such principled stances, I had some trouble following their critique on this point. The authors warn that "because it implies that when there is a conflict law must always prevail, Draft TCPS‐2 embodies a doctrine that represents a significant threat to academic freedom." (5) But they go on to note,

Of the hundreds of thousands of studies in Canada over the past few decades that collected personal information about one or more participants, we are aware of only one researcher who has ever received a subpoena from a third party seeking the disclosure of confidential research information, and he has received three. Three subpoenas in the history of all types of research in Canada. How many times was the subpoenaed researcher ordered to disclose information? Not one. (10)

If subpoenas for confidential information are rarely issued and never enforced, how can a requirement that researchers submit to the law be "a significant threat to academic freedom"?

Perhaps there are situations other than defending confidential information from subpoenas that would put social researchers in conflict with laws, but they are not detailed in the critique. If the policy fails only Russell Ogden, it is certainly a threat to his academic freedom, but a minor threat to academic freedom in Canada in general.

2. TCPS-2 Requires "Anticipating That Which Cannot be Anticipated"

Palys and Lowman complain that "Draft TCPS‐2 sets researchers an impossible task by requiring them to anticipate the unanticipated by requiring them to inform participants what they will do if they make incidental findings." (12)

For example, the TCPS-2 section on informed consent notes that

it is not always possible to anticipate with any specificity the nature of the incidental findings that may surface in the course of research. It is therefore not possible to inform prospective participants in anything but the most general terms of what the research may reveal, beyond the realm of the research question itself.

So, for example, social science researchers embarking on questions of a personal nature should inform prospective participants of the legal obligations they are under to reveal information concerning certain types of abuse. (p. 37, lines 909-916)

The two parts of this quotation contradict each other; the first requires only the "most general" warnings, while the second demands warnings "concerning certain types of abuse" that must be revealed.

Palys and Lowman explain how this contradiction could play out:

Consider what could happen if a Muslim student submitted an application to an REB [research ethics board] to interview other Muslim students about their encounters with non‐Muslims in Canada. Can the REB argue that these university‐age Muslims fit the demographic profile for possible terrorist activity? Can they require the student to put in a caution that, if her subjects tell her anything about terrorist involvements, she will feel obliged to report this to an authority? Criminologists at SFU [Simon Fraser University] sometimes encountered precisely this kind of problem when REB members, none of whom had any experience doing criminological research, sometimes brought outlandish stereotypes to the review process. (14)

Or,

Take, for example, the possibility of “incidentally finding” child abuse in research about coaching children’s sports. At the time of securing consent, the draft policy requires a researcher to say something like the following to each coach: “Even though my research is about coaching and I will treat what you say as confidential, if you tell me about situations where you abuse children, I will have to report you.”

Notwithstanding the way this approach appears to violate the principle of respect for participants — it treats all persons who have any contact with children as possible abusers — the ironic ethical consequence of treating informed consent as an absolute value rather than balancing it with other ethical principles is the harm‐perpetuating consequence of Draft TCPS‐ 2’s imposed limitation on confidentiality. If the researcher declares that he/she will report all incidents of child abuse, the chances are that any child abuser would not divulge that piece of information, with the result that the abuse continues. As abuse is not the focus of the research, an alternative ethical approach would be to say nothing. That way the researcher maintains respect for the participants’ integrity unless there is some concrete reason to do otherwise and would allow him/her to deal with the ethical dilemma of “incidental discovery” of child abuse at the point of discovering it unwittingly. (14)

Palys and Lowman are highlighting a structural problem in the TCPS-2: its assumption that prior review of a project can highlight the most significant ethical dilemmas. Ethics boards may make sense for lab experiments, but the PRE offers no reason to believe that they serve fieldwork. This brings us to the third point:

3. TCPS-2 Lacks a Factual Basis

Palys and Lowman note that

At least REBs have to justify their decisions. By contrast, Draft TCPS‐2 too often imposes “right answers” that do not reflect the diversity of the research enterprise, and offers no explanation for its recommended changes. To facilitate feedback, PRE should have provided an annotated draft. Because of its failure to do so the first time around, PRE should produce an annotated second draft TCPS‐2 explaining why it is recommending certain changes and inviting a second round of commentary . . . Can PRE provide any examples of incidental findings in social science research that resulted in a researcher violating confidentiality? If it cannot, why does Draft TCPS‐2 not walk its own talk by considering the degree of risk that is driving its approach to incidental findings? (15)

Worse still, while offering doubtful solutions to problems that may not exist, TCPS-2 fails to offer solutions to a problem that has been relatively well-documented: abuse of power by by research ethics boards. Palys and Lowman write that the Social Sciences and Humanities Research Ethics Special Working Committee's 2004 report, Giving Voice to the Spectrum,

reported many examples of social science researchers facing REB ethics creep and experiencing infringements of academic freedom as a result of REB activity that had little or nothing to do with ethics. Why does Draft TCPS‐2 spend considerable time outlining putative limitations to confidentiality for threats that have never occurred while doing nothing about REB policy violations and infringement of academic freedom that SSHWC documented? (17)

Though TCPS-2 provides some appeals, "the proposed appeal process also is highly inequitable: an REB can say 'no' in five minutes, while the appeal process can take a year or more, during which time the REB continues to function — and may continue to contravene policy — while the researcher is unable to proceed."

Unlike the authors of TCPS-2, who may have been presenting hypothetical harms, Palys and Lowman write from their own experience and that of their students.

In raising these questions, Palys and Lowman point to a much larger problem with TCPS-2 and its analogues in other nations: a lack of empirical investigation. TCPS-2 makes vague gestures toward empirical grounding, as when it claims that "history offers unfortunate examples where participants in research have been needlessly and at times profoundly harmed by research." (p. 1, lines 21-22) Maybe so, but good historians cite their sources, and the authors of TCPS-2 have not.

Sound policy relies on sound research. As Palys and Lowman note, there is nothing in the TCPS-2 to
suggest that its authors are sufficiently familiar with the real problems faced by social scientists to offer real solutions. They complain,

PRE’s strategy is that of an ethics deity imposing its own “right answers” rather than fulfilling its mandate to educate, promote discussion, respect disciplinary and methodological diversity, build consensus, and cultivate a culture of research ethics in Canada. (21)

If PRE is a deity, it is not omniscient. Let us hope it is not omnipotent either.

Less Flexibility, More Freedom

2009-02-14T14:43:00.000-05:00

Defenders of the present IRB system often boast of the "flexibility" offered by current regulations. (See, for example, Dr. Jeffrey Cohen's report from the November PRIM&R meeting.)

But flexibility--when combined with the possibility of punishment--can actually empower censorship. Here is how Human Rights Watch describes an analogous system, China's censorship of the Internet:

The display of politically objectionable content can result in reprimands to company management and employees from the MII, the State Council Information Office, the Communist Party's Propaganda Department, and/or various state security organs, accompanied by warnings that insufficient controls will result in revocation of the company's license. In order to minimize reprimands and keep their licenses in good standing, BBS and blog hosting services maintain lists of words and phrases that either cannot be posted or which cause monitoring software to "flag" the content for manual removal by employees.

Search engines likewise maintain lists of thousands of words, phrases and web addresses to be filtered out of search results so that links to politically objectionable websites do not even appear on the search engine's results pages, even when those websites may be blocked at the backbone or ISP level . . . Such lists are not given directly to Internet companies by the Chinese government; rather, the government leaves the exact specifics and methods of censorship up to companies themselves. Companies generate their "block-lists" based on educated guesswork plus trial-and-error: what they know to be politically sensitive, what they are told in meetings with Chinese officials, and complaints they may receive from Chinese authorities in response to the appearance of politically objectionable search results.

But the complicity of companies is even more direct: they actually run diagnostic tests to see which words, phrases, and web addresses are blocked by the Chinese authorities at the router level, and then add them to their lists, without waiting to be asked by the authorities to add them. And because they seek to stay out of trouble and avoid complaints from the authorities, many businesspeople who run [Internet Content Providers] in China confess that they are inclined to err on the side of caution and over-block content which does not clearly violate any specific law or regulation, but which their instincts tell them will displease the authorities who control their license. In all these ways, companies are doing the government's work for it and stifling access to information. Instead of being censored, they have taken on the role of censor.

In other words, by keeping secret the exact terms that will trigger a license revokation, the Chinese government achieves more censorship than it could by publishing a list of forbidden terms, and it makes Google, Yahoo!, and other U.S. companies complicit in the censorship. Similarly, OHRP's vagueness about what will trigger a shutdown fails to assure universities that they can safely deregulate research, so universities restrict research that should be exempt from review.

Fortunately, OHRP's new director, Jerry Menikoff, understands this. In the Winter 2009 issue of AAHRPP Advance he writes,

We often hear that it’s better not to provide specific guidance—that the absence of guidance allows people greater flexibility in interpreting the regulations. In my experience, the opposite can be true. Guidance can empower individuals and advance both research and research protections. In the absence of guidance, people tend to be reluctant to take certain actions out of fear that they are violating the rules. In some instances, important research is not even attempted, all because of a misunderstanding. Guidance could eliminate the misconception and clear the way for research.

I'm delighted that Dr. Menikoff takes this approach, and I look forward to more specific guidance from OHRP that would clear the way for research.

[Thanks to Rob Townsend for altering me to Menikoff's comments.]

AAUP's Rhoades Takes Soft Line on IRB Training

2009-02-13T12:19:00.001-05:00

In an essay on compulsory sexual harassment training ("Sexual Harassment and Group Punishment," Inside Higher Ed, 12 February 2009), the new AAUP general secretary, Gary Rhoades, offers side comments on human subjects research training:

In research universities (where professors’ work routinely involves human subjects, though even there literary and some other scholars are not required to undergo such training), perhaps the most obvious example of this is the human subjects training surrounding research grants and activity. Prior to getting grants approved by the sponsored projects division of a university, an investigator must have undergone human subjects training. Although the training varies by university, there are common patterns nationally. Typically, for example, such training is online, and is not particularly rigorous, to put it mildly. Indeed, the format involves investigators taking an exam by reading some written passages and then answering questions about them. After each section or module the person finds out whether he or she missed too many questions in a section, and proceeds. If they have missed too many questions in a section they simply backtrack, get the same questions in a different order, and retake the quiz, until they pass. A widely used set of exams (which are specified to social/behavioral and biomedical research) are those offered by the Collaborative Institutional Training Initiative, which over 830 institutions and facilities (including a very large number of research universities, and indeed including the University of California at Irvine) utilize. The modules for the CITI quiz typically include three to six questions.

For the most part, although faculty complain about the inconvenience and irrelevance of the training, I do not know of anyone who would suggest that such training should be required only of investigators found to have violated the rights of human subjects. The more important questions of process and principle surround the institutional review board activities that regulate the approval of an investigator’s proposal. Here, serious questions have been raised about compromising investigators’ academic freedom to engage in certain types of research and to research certain subject matter. But the controversy is not, for the most part, about the human subjects training per se. Indeed, I would venture to say that for colleagues in the social and behavioral sciences, among the most common comments and complaints about human subjects training are that it is ineffective, that it does little by way of actually protecting human subjects and seems to be geared more to protecting the institution.

Apparently, Dr. Rhoades is unfamiliar with the widespread, principled opposition to CITI and other online training programs. That is worrisome, if it signals the retreat of AAUP from its longtime leadership in the fight against overly broad human subjects regulations and requirements.

More Comments on Maryland's IRB Law

2009-02-10T21:12:00.000-05:00

In January 2008, I commented on Adil E. Shamoo and Jack Schwartz, "Universal and Uniform Protections of Human Subjects in Research," American Journal of Bioethics 7 (December 2007): 7-9. That essay applauded a 2002 Maryland law that seeks to impose federal regulations on all human subjects research conducted within the state, even if not conducted at an institution with an FWA. As I noted at the time, the law is of dubious constitutionality, and it has probably survived this long only because it has never been enforced.

The November 2008 issue of the same journal reprints the Shamoo/Schwartz essay as a "target article," along with eight invited commentaries. Several of the commentaries complain that the federal regulations--and therefore the Maryland law--are insufficiently protective, rather than overly broad, but none of these address social science.

Three commentators do address Shamoo and Schwartz's failure to consider the impact on social science research. Neil W. Schluger complains that "their solution will do very little to protect human subjects, and perversely, it may actually make the situation worse by simply piling more studies into an overburdened and flawed system." (13) He notes that

a very large percentage of studies that IRBs review are studies involving minimal risk to subjects. Many of these are observational studies, reviews of existing data, studies in which the only intervention is administration of a questionnaire, or other types if studies where there really is no reasonable expectation that any harm could result. Although such studies can be reviewed by the use of expedited review procedures without convening the full IRB, they still require considerable administrative and regulatory oversight by IRB staff. Further efforts should be made to reduce the work associated with these harmless studies. (14)

David B. Resnik notes that the idea of regulating all research nationwide has been kicking around since 1995, and has been the subject of six failed bills in Congress. But, he continues, "it is not obvious that society would gain much by requiring an organization, such as Gallup or ABC News, to submit a proposal to an institutional review board (IRB) to conduct an anonymous survey each time that it decides to gauge public opinion on a particular issue. Social resources might be better spent overseeing riskier research, such as clinical trials." (6) He concludes that "Any proposal that is made into a law should include adequate provisions for exempting some low risk research and clarifying the definitions of important terms." (8)

Lisa M. Rasmussen complains that the Shamoo/Schwartz position "makes no distinction between highly risky biomedical research and, on the opposite end of the spectrum, research that involves no more risk than we all accept daily. Does beneficence really require that human subjects be protected from answering questionnaires or being interviewed? If so, why does this protection not extend to marketing, polling, or journalism?" (18)

Instead, Rasmussen proposes

the one-time approval of research “templates.” Taking advantage of the fact that a great deal of research follows traditional disciplinary methods, this model suggests that IRBs could approve a variety of research templates (written, for example, by discipline-specific bodies such as the American Psychological Association (Washington, DC), or by a researcher whose classes may repeat experiments semester after semester), and grant automatic exemption to any researcher using such templates. Accountability and oversight could be ensured by requiring the researcher to submit a simple form to the IRB agreeing to use such a template (which would also include provisions for protecting confidential data). Were this form electronically based, research could proceed as soon as the form was submitted, without requiring submission of a protocol or awaiting approval. This achieves the goals of both minimizing bureaucracy and protecting human subjects. It meets our moral obligations to human subjects of research without uniformly requiring IRB oversight of research. (18)

Rasmussen concludes that "universal and uniform regulation of all human subject research is well-meaning but un-nuanced." (18)

It is a pity that the journal does not print a reply by Shamoo and Schwartz. As I noted in my post last year, Shamoo himself has suggested that what he terms "low-risk research" is overregulated. I remain puzzled why he favors state laws that promise even more regulation of such research.

Oral History Wins and Loses

2009-02-06T23:33:00.001-05:00

Two large public universities have recently announced policies concerning IRB review of oral history. It was a split decision.

The University of Nebraska-Lincoln (UNL) promulgated Policy # 001: IRB Review of Oral History Projects, dated 1 October 2008. The policy is based on Columbia University's 2007 oral history policy. According to the Nebraska policy,

Oral history interviews, that only document specific historical events or the experiences of individuals or communities over different time periods would not constitute “human subjects research” as they would not support or lead to the development of a hypothesis in a manner that would have predictive value. The collection of such information, like journalism, is generally considered to be a biography, documentary, or a historical record of the individual’s life or experience; or of historical events. Oral history interviews of individuals are not usually intended to be scientific or to produce generalizable information and hence are not usually considered “research” in accordance with the federal regulations or UNL policy. Therefore, such oral history activities should not be submitted to the UNL IRB for review.

The explanatory memo borrows two explanatory examples from Columbia, without attribution. It does cite OHRP's 22 September 2003 letter to the Oral History Association and American Historical Association, in which Dr. Michael Carome promised that "most oral history interviewing projects . . . can be excluded from institutional review board (IRB) oversight because they do not involve research as defined by the HHS regulations."

So far so good. But less than four months later, the University of Illinois at Chicago (UIC) released a policy statement, "IRB Review of Oral History and Other Social Science Projects" and an accompanying Tip Sheet: IRB Review of Oral History and Other Social Science Projects." This policy demands that "investigators wishing to perform oral histories or other social science projects must complete and submit the Determination of Whether an Activity Represents Human Subjects Research form to the [Office for the Protection of Research Subjects]."

UIC claims that "increasingly . . . the application of qualitative research methodologies may render studies that typically would not have required IRB review and approval to be submitted for IRB review or, at least, to require a determination from the IRB as to whether the study is subject to human subjects protection regulations." If anyone at UIC can show me that this statement is based on a study of trends in historical scholarship (rather than being made up to justify the policy), I will send a small box of good chocolates.

Follow a few links, and you can find the real basis for the UIC policy: Dr. Carome's December 2003 e-mail to Lori Bross of Northern Illinois University, in which he backed way from his September 2003 promise to the historians and embraced a set of nonsensical examples first proposed by the UCLA Office for Protection of Research Subjects.

In short, OHRP's perfidy of 2003 still echoes across the country. Depending on whether you believe Michael Carome's statements of September 2003, October 2003, December 2003, or January 2004, you may come away with very different understandings of the federal government's policy toward oral history, and the leeway allowed to institutions.

PRIM&R Plans SBER Webinar

2009-02-05T22:51:00.000-05:00

On February 25, PRIM&R will host a "webinar" entitled "New Solutions to Ongoing Problems When Reviewing Social, Behavioral, and Educational Research." The two faculty are Mary Marshall Clark, who successfully liberated oral history from IRB oversight at Columbia, and J. Michael Oakes, whose chapter in Institutional Review Board: Management and Function more or less claims that questionnaires can drive people to suicide. That's an interesting contrast, though not interesting enough for me to pay $175 to participate.

And the Bush Was Not Consumed

2009-01-31T16:47:00.003-05:00

Robert Dingwall kindly alerted me to Scott Kim, Peter Ubel, and Raymond De Vries, "Pruning the Regulatory Tree," Nature 457 (29 January 2009), 534-535.

The authors have complex disciplinary backgrounds. Kim has an MD and a PhD in philosophy, and teaches psychiatry. De Vries has a PhD in sociology and the title of associate professor of bioethics/medical education. And Ubel has an MD and describes himself as a "physician and behavioral scientist." Together, they are unusually well prepared to think about the full range of disciplines affected by human subjects regulations.

The authors complain that IRB review of minimal risk research wastes time and money, infringes academic freedom, and threatens public health by impeding important research. They conclude that "it is unethical to support a system that creates a significant financial, scientific, clinical and ethical burden with virtually no counterbalancing good."

To fix this, they propose "a simple regulatory change that is far-reaching, equitable and yet low risk: exempt minimal-risk research from IRB review." In their scheme, researchers would still "complete a brief application describing research procedures, risks, burdens and the potential loss of otherwise expected benefits to the subjects. An institution-designated person reviews the application, and exempted protocols would not be subject to further IRB review. The application becomes the project's registration and serves as an accountability document."

This sounds fine on the surface, but the proposal raises some questions.

1. How can we change the Common Rule?

The authors propose a change in the regulations, but they do not address the difficulty of getting agreement from all Common Rule signatories. In 2002, Marjorie Speers of AAHRPP testified that " there is no effective means to [revise policy]; the agencies who are signatories to the Common Rule have not been able to make changes to it in the last 11 years even though the need for changes has existed." That was nearly seven years ago, and nothing has improved. See The Calcified Common Rule.

Under the circumstances, we may need new legislation, rather than regulation.

2. Would OHRP and institutions respond to changed regulations?

The authors recognize that even if the regulations were changed, "institutions have a tendency to impose on themselves requirements that are even more stringent than those required by law," so they might impose review on minimal-risk research anyway. Indeed, the 1998 Bell study found that less than half of protocols eligible for exemption actually received it. I imagine that percentage has gone down in the past decade, considering regulators' hostility to the existing exemptions.

To address this problem, the authors claim that "a new regulation that exempts minimal-risk research from IRB review would send a clear and unambiguous message that the government's priority is not on intense oversight of low-risk research." Perhaps, but the government does not speak with one voice, and regulators have resisted earlier calls for deregulation. The greatest effort at deregulation--the 1981 exemptions--was followed within months by OPRR suggestions that universities act more stringently than required by the regulations. The 1995 ACHRE recommendation for "alternative mechanisms for review and approval of minimal-risk studies" was followed by the OPRR crackdown that led to far more burdensome review of minimal risk research.

If the authors are serious about reform, they should seek not a "message," but rather a law that restrains OHRP and penalizes interference with minimal risk research.

3. Would exemption be any better than expedited review?

As noted above, when the authors call for "exemption," they use the term in the specialized way introduced by OPRR in 1995. Rather than meaning that a researcher is free from IRB jurisdiction, they mean that a researcher would still have to write up an application and submit it to an "institution-designated person." The authors claim that "substantial resources would be freed for better uses."

The authors base this claim in part on a 2005 finding that in US medical schools, the average expedited review cost more than the average full review ($1,060 vs. $1,021). [See Jeremy Sugarman, Kenneth Getz, Jeanne L. Speckman, Margaret M. Byrne, Jason Gerson, and Ezekiel J. Emanuel, "The Cost of Institutional Review Boards in Academic Medical Centers," New England Journal of Medicine 352 (April 28, 2005), 1825-1827.] But that study did not estimate the cost of reviews for exemptions.

Indeed, the system the authors propose is not very different from the current system for expedited review; the big difference being that the authors would delegate authority to an "institution-designated person" who might not be an IRB member. But their proposal would still require a good deal of paperwork and staff time, as well as the review by a single person, so it might not cost much less than the expedited system they seek to replace.

This question is open to empirical research. The authors, or someone else, could repeat the Sugarman analysis, but studying institutions that require staff review for proposals that are exempt under the existing 45 CFR 46.101 clauses.

4. Is pruning enough?

In sum, the regulatory tree may be tougher to prune, and to keep pruned, than the authors suppose. Here's a counter proposal adapted from advice concerning a similar growth:

For successful long term control of human subjects regulations, the extensive root system must be destroyed. Any remaining root crowns can lead to reinfestation of an area. Mechanical methods involve cutting vines just above ground level and destroying all cut material. Close mowing every month for two growing seasons or repeated cultivation may be effective. Cut human subjects regulations can be fed to livestock, burned or enclosed in plastic bags and sent to a landfill.

Blame the ESRC?

2009-01-27T20:41:00.002-05:00

David Hunter kindly alerted me to Sarah Dyer and David Demeritt, "Un-Ethical Review? Why It Is Wrong to Apply the Medical Model of Research Governance to Human Geography," Progress in Human Geography 33 (2009).

Dyer and Demeritt attack the Economic and Social Research Council's 2005 Research Ethics Framework, the basic document guiding British Research Ethics Committees (RECs) in their oversight of social research. Some of this attack strikes me as misplaced, since it ignores elements of the Framework that address the authors' concerns.

For example, the authors complain that

in the case of critical social science, the aim of the research is typically to expose wider social injustices and in that way actually harm those who benefit from them. But, following the injunction of the ESRC (2005: para 3.2.5) that ‘[p]articipants’ interests or well-being should not be damaged as a result of their participation in the research’, it would be impossible to secure permission to interview employers whose discriminatory practices a researcher was hoping to expose and thereby end. (55)

But the ESRC understands this (somewhat), noting,

Much social science research has a critical role to play in exploring and questioning social, cultural and economic structures and processes (for example relating to patterns of power and social inequality), and institutional dynamics and regimes that disadvantage some social groups over others, intentionally or not. Such research results may have a negative impact on some of the research subjects. (22)

Likewise, Dyer and Demeritt write,

there are times when safety, either of the investigator or of research subjects themselves, means that research subjects cannot be informed about the true nature of research [such as] studies of human trafficking, illegal workers’ gang masters, and so on. In such cases the notion of asking participants to sign a consent form of the sort envisioned by the ESRC (2005) is ridiculous, and the ill of deception balanced by the importance of the research and a commitment to protecting anonymity. (57)

But the Research Ethics Framework recognizes this problem, stating, "informed consent may be impracticable or meaningless in some research, such as research on crowd behaviour, or may be contrary to the research design, as is often the case in psychological experiments where consent would compromise the objective of the research. In some circumstances – such as users of illegal drugs – written consent might also create unnecessary risks for research subjects." (21)

The real problem, it seems, is not the ethical content of the Research Ethics Framework, but the structure it employs to promote those ethics. Project-by-project committee review may be unable to handle the complexity and unpredictablity of social science research. As Dyer and Demeritt write,

Whereas drugs trials involving vast sums of money, or biomedical research on extremely vulnerable people in enormous pain need only gain anticipatory approval, the ESRC argues that because ‘purposes, methods, and intended uses’ of qualitative research evolves as it proceeds, this kind of research should be required to seek REC approval on multiple occasions. Such a disproportionate response is itself unethical if it overburdens researchers such that worthwhile research does not get done. (57)

The underlying issue is that committee review is a process specifically tailored to experimental research. The more a researcher fits the experimental model, the more detailed her protocol will be, and the more amenable to prior review. It's the open-ended, qualitative researcher who has the low signal-to-noise ratio, and for whom ethics committee represent such a waste of time.

IRB Says Women Can't Be Photographed

2009-01-26T22:50:00.002-05:00

The New York Times reports IRB interference with a study that combines medical and social research. (Gina Kolata, "Fitness Isn’t an Overnight Sensation," 21 January 2009).

Carl Foster, an exercise physiologist at the University of Wisconsin, La Crosse, was amused by ads for a popular piece of exercise equipment. Before-and-after photos showed pudgy men and women turned into athletes with ripped bodies of steel . . .

“We said: ‘Wait a minute. You can’t change yourself that much,’ ” Dr. Foster said. So he and his colleagues decided to experiment. Suppose they recruited sedentary people for a six-week exercise program. Would objective observers notice any changes in their bodies?

The plan was to photograph volunteers wearing skimpy bathing suits and then randomly assign them to one of three groups: cardiovascular exercise, weight lifting or control. Six weeks later, they would be photographed again.

Their heads would be blocked out of the photos, which would be shuffled. Then the subjects and judges would rate the body in each photo on a scale of 1 to 10, with 10 being spectacular.

The volunteers were men, age 18 to 40 (the university’s human-subjects review board looked askance at having women photographed and rated like that).

This is another case where Canada's TCPS (even the existing version) shows a more sophisticated undertsanding of research ethics than does the Belmont Report or the U.S. regulations. Whereas the Belmont authors were concerned about vulnerable populations being inappropriately targeted for medical studies, they did not think about the arbitrary exclusion of populations, or the stigmatization of competent adults as incompetent. By contrast, Article 5.2 of the 2005 version of the TCPS states that "women shall not automatically be excluded from research solely on the basis of sex or reproductive capacity."

I am unaware of any U.S. regulation or guidance that so specifically forbids the kind of sexism displayed by the La Crosse IRB. Still, it's disappointing that that IRB thinks adult women incapable deciding for themselves whether to participate in a study open to men.

For this study's implications for advertising law, see Rebecca Tushnet's 43(B)log.

Canada Considers a New TCPS

2009-01-22T21:53:00.003-05:00

The Research Ethics Blog alerted me to the relase of the Draft 2nd Edition of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS), Canada's manifesto of research ethics. The document is strong in its recognition of critical inquiry. But it is weak in establishing mechanisms to meet its stated goals of proportionate review and reasoned decision-making. It fails to address some of the key concerns in the 2004 report, Giving Voice to the Spectrum, and threatens to leave social scientists not much better off than they are today.

Valuing Critical Inquiry

The new draft recognizes differences among the various activities that go by the name "research." Like the Belmont Report, it offers three core principles: concern for welfare, respect for autonomy, and respect for the equal moral status of all humans. Unlike Belmont (and particularly unlike 45 CFR 46), it recognizes that

these principles are not absolute. They may, at times, conflict. They do not apply in all circumstances, to all types of research, as is set out in the following chapters. How they apply and the weight to be accorded to each one will depend on the nature and context of the research being undertaken. (2)

Such insight is elaborated in sections on social research. In particular, the draft Statement embraces "critical inquiry," in passages longer and stronger than those in the existing Statement:

Research based on critical inquiry – focusing, for example, on public policy issues, modern history, or literary or artistic criticism – may involve interaction with living individuals, notably through interviews. Where the aim of the researchers is to engage in a critical examination of a body of artistic work, a public policy, other comparable types of work, the role of the REB should be limited to ensuring that researchers conduct their work respecting the professional standards of their discipline(s) or field(s) of research. The need to ensure freedom of inquiry and to protect the ability of researchers to criticize the work (or organization, political party, corporate enterprise, etc.) they are examining takes precedence over the need to protect individual parties from harm. (9)

Alas, the authors of the draft are inconsistent in their understanding of the role of critical inquiry. They write,

In certain areas of research in the social sciences and humanities, such as political science, economics or modern history (including biographies) . . . the purpose of the research may be to cast a critical eye on organizations, political institutions, or systems or individuals in public life. The outcome of these types of research may harm the reputation of public figures or institutions in politics, business, labour, the arts, or other walks of life. Such harm may, however, be an unavoidable outcome of research that seeks to shed light on or to critically assess the work of a public figure or institution. Where the purpose of the research is to advance knowledge about the workings, for example, of a public office or a public figure, the risk–benefit analysis by the REB should focus on whether the approach they have adopted respects the professional standards of the researcher’s discipline or fields of research. Just as a bruise is an unavoidable risk of research that requires a needle-stick, so harm to reputation is an unavoidable risk of certain types of social science inquiry, and it must be treated as such. (14)

That's fine until the last sentence, which suggests that the authors don't understand critical inquiry at all. A bruise is an unfortunate byproduct of a needle-stick. A harm to reputation is often the deliberate objective of critical inquiry. The trick, William Potter, is not minding that it hurts.

Seeking Proportionality

The draft Statement concedes that too much review can be a bad thing, stating that

The scope and intensity of ethics review should be proportionate to the level of risk involved. When those involved in the review of research tailor their level of scrutiny to the level of risk, they reduce unnecessary impediments and facilitate the progress of worthwhile and ethical research. This is the crux of proportionality, and it is a message that recurs throughout this Policy.

It is equally important that ethics review be appropriate to the disciplines, fields of research and methodologies of the research being reviewed. This means that REBs must understand the discipline and methodology under review and be able to assess the research on its own terms. (5)

To give researchers a fair shot, the Statement insists that "at least two members should have the relevant knowledge and expertise to understand the content area and methodology of the proposed or ongoing research, and to assess the risks and benefits that may be associated with the research," and that it seek ad hoc advisors when necessary. (58) It even requires "an established mechanism and procedure in place for entertaining appeals" (69), even permitting ad hoc appeals board, something forbidden by the existing statement.

Unfortunately, the draft does little to ensure "a proportionate approach to ethics review" or even to consider what that means. There's nothing in the statement to suggest that ethics boards are uniquely suited to uphold ethical standards, or that they are necessary for all kinds of human subjects research. So why must all research pass through an REB? While the idea of proportionality recurs throughout the document, when we get to its actual application (p. 63) we find only two levels of review: delegated REB review of minimal-risk research, and full REB review of everything else. A two-speed transmission is not very proportionate.

Giving Voice to the Spectrum called for "different approaches to ethics review that would allow REB blanket approval of programs of research based on the overall ethics strategy of the researcher (or team of researchers), within specified parameters" as well as "exemptions from review for social science and humanities research that involves standard practice in the discipline involved." (6)

Along these lines, a more proportionate system would allow for several levels of review, e.g.,

1. No review--for activities even a middle-schooler can do (e.g., conversations with family members)
2. Researcher certification--for activities where the key thing is to be sure the researcher is familiar with professional standards. The draft itself suggests that this is appropriate for much interview research.
3. Departmental committee review.
4. Delegated review.
5. Full board review.
6. Full board review plus outside consultation--for the riskiest research.

Another problem with proportionality is its dependence on the threshold of "minimal risk," which doesn't map well onto the type of critical inquiry that the policy seems to endorse. As noted above, the draft states that "where the aim of the researchers is to engage in a critical examination . . . the role of the REB should be limited to ensuring that researchers conduct their work respecting the professional standards of their discipline(s) or field(s) of research." That would seem to imply that the intent of the research, as well as its level of risk, should determine the level of review.

Ignoring Evidence?

Another big concern is that the document makes no requirement that REBs base their decisions on empirical reality. Giving Voice to the Spectrum called for "a shift in onus where, in order to require changes to a research proposal, an REB would be obliged to explain what identifiable harm has not been addressed, and how their proposed solution will ameliorate the problem." (6) That has not happened.

Article 6.13 of the draft provides that "the research ethics board shall function impartially, provide a fair hearing to those involved and provide reasoned and appropriately documented opinions and decisions." But what does "appropriately documented" mean here?

The 2005 TCPS also required "reasoned and well-documented decisions" (Article 1.9). But much conflict between REBs and researchers takes place when they cannot agree on what is meant by reasoned and documented. For example, when Tony Porter wanted to do some interviews about the governance of research ethics, his REB demanded that it be allowed to scrutinize the questions in advance, looking for some mysterious risk. Would such behavior count as a fair hearing with reasoned and appropriately documented opinions and decisions? Are boards empowered to dream up fantastic risks? Or should the standard be higher, demanding that ethics boards operate according to scholarly standards, and document the risks that they attribute to a project?

Again, compared to the Belmont Report, the ethics sections of this new TCPS are relatively nuanced, reflecting long thought and the input of a great many scholars. I'd like to see American universities list the TCPS as the "statement of principles" on their Federalwide Assurances. (See The Dormant Right to Ethical Self-Determination".) But by assuming the need for board review, and by failing to set standards for board decision-making, the draft leaves research vulnerable to the "unnecessary impediments" it seeks to eliminate.

OHRP to Revise Website

2009-01-10T23:33:00.002-05:00

OHRP has called for feedback on the "content, format, navigation or any other aspect" of its website. Below are the comments I sent in.

Dear OHRP,

Thank you for your invitation for suggested improvements to the OHRP website. Before making any suggestions, I must first thank you for the existing website, which contains a great deal of extremely valuable information. As a historian, I particularly appreciate the historical material, including the Belmont Report oral history section, and the compilation of key notices from the Federal Register dating back to 1973.

With this in mind, let me suggest some additional material that would be helpful.

1. Responses to calls for comments

When OHRP issues calls for comments in the Federal Register, it routinely advises that "comments received within the comment period, including any personal information provided, will be made available to the public upon request." While I am grateful to OHRP staff for sending me electronic versions of these comments in replies to my requests, I suggest that all comments be posted on the OHRP website as well. The comments I have seen constitute thoughtful, informed positions on the regulation of human subjects research and mark significant contributions to the debate on the subject. They have been the subject of reporting in the press, suggesting the wide interest in them.

Placing the comments on the website would insure equal access to those comments by all concerned, and would solve the problem of electronic files that are too big for easy transmission by e-mail. Posting would also preserve the comments for future readers, a key concern given that many issues crop up repeatedly. For example, in 2007, OHRP requested comments on expedited review. Anyone responding to that call would have benefited from access to the comments received in reply to the similar call of 1997. But those earlier comments seem to have been lost to history.

2. Reports to which OHRP has contributed

In June 2008, the Human Subjects Research Subcommittee published a report called "Expedited Review of Social and Behavioral Research Activities." Perhaps this is not technically an OHRP report, but the Subcommittee was co-chaired by the acting director of OHRP, and both he and Mr. Drew of OHRP were members of the working group that produced the report. Yet there is no link to the report on the OHRP website, nor did I receive notice of it from the OHRP-L@LIST.NIH.GOV mailing list. When OHRP staff, acting in their official capacities, contribute to a report like this, it should be announced on the list and a link added to the OHRP site.

3. A complete copy of the Bell Report

Your current page, http://www.hhs.gov/ohrp/related.html, includes a copy of "Evaluation of NIH Implementation of Section 491 of the PHS Act, Mandating a Program of Protection for Research Subjects," published in 1998. But it does not include the technical appendices mentioned on page ii of that report, nor have I been able to locate a copy of those appendices. Posting them would be of value.

4. A link to OPRR Reports

The section on "Historical Documents" on the page http://www.hhs.gov/ohrp/related.html does not mention the OPRR Reports posted at http://www.hhs.gov/ohrp/dearcoll.htm. As the latter page indicates, most of these documents exist "solely as a reference for historical purposes." I am very glad that these documents are online, but I suggest that as historical documents, they deserve a link under the "historical documents" section. For that matter, I'd like to see the "Historical Documents/Information" section, including the Federal Register notices, given its own page, but I realize this wish may reflect my unusual perspective as a historian. I would also like a note added clarifying if the OPRR Reports now online represent all or just some of the OPRR Reports sent out during that office's existence.

Thank you again for this opportunity to comment on the website. If I think of additional suggestions, I will be sure to send them in.

Sincerely,

Zachary M. Schrag
Assistant Professor of History, George Mason University

Happy New Year, OHRP

2009-01-01T20:00:00.002-05:00

In February 2007, the New York Times reported on scholars' frustration with IRB review of social research. OHRP assured reporter Patricia Cohen that it was working on the problem:

Bernard A. Schwetz, director of the federal Office for Human Research Protections, which administers the regulations, acknowledges that the guidelines covering the boards’ actions have not been clear enough and says he intends to make public new proposed guidelines before the end of the year.

Later in the article, Schwetz "said the new guidelines 'will give a lot of examples and will give more guidance on how to make the decision on what is research and what is not.'" This makes it clear that the 2007 call for comments on expedited review is not what he had in mind.

With the end of 2008, the promised guidelines are more than a year late.

This may be for the best. Schwetz seemed determined to continue OHRP's history of trying to regulate everything, while under Menikoff, we might expect better results. But OHRP and its predecessors have made a lot of empty promises over the years, and it's frustrating to see that tradition continued.

The Costs of Ethical Review

2008-12-25T13:35:00.002-05:00

In his article on "Regulatory Innovation," discussed here earlier, Scott Burris complains that

the core problem with the Common Rule is the IRB’s power to treat its insights and risk–benefit calculations as ‘‘right answers’’ that may be imposed at no cost to the IRB upon researchers whose own ethical reflection may have led to different, equally defensible conclusions.

Robert Dingwall concurs in his essay, "The Ethical Case Against Ethical Regulation in Humanities and Social Science Research," 21st Century Society 3 (February 2008): 1-12. Though Dingwall is British, he notes that the system there looks "very like US Institutional Review Boards, and their analogues in Canada and Australia." (4) British boards, and British rules in general, fail to account for the costs of ethical review.

This has real consequences. Dingwall relates his own experience:

A colleague and I were recently commissioned by the NHS [National Health Service] Patient Safety Programme to study the national incidence and prevalcence of the reuse of single-use surgical and anaesthetic devices, and to consider why this practice persisted in the face of strict prohibitions. Part of this involved an online survey, using well-established techniques from criminology to encourage self-reporting of deviant behaviour, so that relevant staff in about 350 hospitals could complete the forms without us ever needing to leave Nottingham. However, a change in NHS ethical regulation meant that we needed approval from each site, potentially generating about 1600 signatures and 9000 pages of documentation. Although we never planned to set foot in any site, it would also have required my colleague to undergo around 300 occupational health examinations and criminal record checks. As a result, we were unable to carry out the study as commissioned and delievered a more limited piece of work. Other estimates suggest that the practice we were studying leads to about seven deaths every year in the UK and a significant number of post-operative infections. The ethical cost of the NHS system can be measured by the lives that will not be saved because our study could not investigate the problems of compliance as thoroughly as it was originally designed to. (10)

This is a stark example, but Dingwall sees it as emblematic of a general drag on social research that has consequences for the future of free socieites. Ethical regulation of humanities and social science research, he argues, contributes to "a waste of public funds, serious information deficits for citizens, and long-term economic and, hence, political decline . . . " (10)

Dingwall discounts the need for oversight, arguing that humanities and social science researchers "do nothing that begins to compare with injecting someone with potentially toxic green stuff that cannot be neutralised or rapidly eliminated from their body if something goes wrong. At most there is a potential for causing minor and reversible emotional distress or some measure of reputational damage." (3) I think this takes the case too far. See Sudhir Venkatesh’s Gang Leader for a Day for a recent example of a social scientist who seriously hurt people by breaking their confidences. (The book is recent; the incident took place in the early 1990s.) Dingwall's own research, had it exposed a physician who was illegally re-using devices, would have done irreversible harm to that physician. Rather than arguing that such harms are impossible, Dingwall would be better off arguing that they are a) rare, and b) not likely to be prevented by the forms of prior review now in place.

The Belmont Report calls for "systematic, nonarbitrary analysis of risks and benefits . . . This ideal requires those making decisions about the justifiability of research to be thorough in the accumulation and assessment of information about all aspects of the research, and to consider alternatives systematically." If we were to hold regulatory regimes to the same standard, we would find ample risks, few documented benefits, and no consideration of alternatives.