Hidden Gem III: What’s Wrong with Science?

The final hidden gem is this article on reforming science. Although it appears in the journal Infection and Immunity it is concerned with the culture of science as a whole and will be of interest to all scientists. The authors Arturo Casadevall and Ferric Fang diagnose some serious systemic problems with 21st century science and prescribe some reforms by way of treatment. At the bottom of this article there is a survey for you to express your views.

A strongly encourage you to read the full article but here is a potted summary of the diagnosis and prescription (the evidence to back each point can be found in the original paper):

The diagnosis

1. Workforce imbalance: The scientific career structure is in need of reform. “… the current scientific workforce resembles a pyramid scheme with a small number of principal investigators presiding over an army of research scientists, postdocs, students, and technicians who have little autonomy and increasingly uncertain career prospects.”

2. Publish and still perish: Pressure to publish (rather than discover, understand and explain) is leading to inefficiencies. “Much of the recent scientific literature is repetitive, unimportant, poorly conceived or executed, and oversold; perhaps deservingly, much of it is ignored. However, the sheer number of papers generates an enormous burden on the peer review system.”

3. Survival of the fittest: Targets and competition are distorting science. “The entire shape of the research effort is distorted as researchers scramble to conform their work to targeted funding opportunities and steer away from risky lines of inquiry or projects requiring a lengthy time investment.”

4. Winner takes all: Rewards follow success which can lead to ingenuity but also can lead to a “Matthew Effect” (“for whosoever hath, to him shall be given”) and can encourage cheating and fraud.

5. The priority rule: “The effort to get there first and grab the largest share of the credit undoubtedly contributes to such practices as citation bias, secrecy, and the appropriation of others’ ideas and data.”

6. Science as a team sport: “Credit is disproportionately awarded to principal investigators for what is truly the product of teamwork, and nearly all scientific contributions are heavily dependent on knowledge obtained earlier.”

The prescription

1. Revising criteria for promotion. The authors recommend a “promotion process based on careful peer evaluations of scientific quality and the specific contributions of the authors might help to reduce the present emphasis on priority.”

2. Reembracing philosophy. They argue that philosophy needs to play a more central role in science:
“scientific training today does not include significant instruction in philosophy despite the critical importance of the philosophical branches of logic, epistemology, and ethics to science…  One common error in science is the attempt to make positive inferences from negative data (e.g., ruling out a mechanism or cause and effect from negative experimental data). Errors in logical thought can lead to dogma and affect the direction of entire fields of study.”

3. Enhanced training in probability and statistics.
They suggest that despite superficial respect for the discipline, scientists’ knowledge and application of principles of statistics is rather patchy. “Although most scientists have some knowledge of probability and statistics and can calculate “P values” using statistical software, the level of statistical expertise varies greatly among individuals. In fact, much deeper knowledge of the foundations of these disciplines is needed.”

4. Use of checklists.
This is a straightforward recommendation to adopt a procedural fix which has been beneficial in other fields: “There is conclusive evidence that the use of checklists can reduce errors in human activities ranging from aviation to surgery. Science should be no exception.”

5. Is science too masculine?
They argue that cut-throat winner-take-all competitive culture may be “male evolutionary strategies that disproportionally reward risk taking” and cheating. “A less ‘masculine’ scientific culture could be a fairer, more honest, more cooperative, and more successful enterprise.”

6. Brilliant individuals do not necessarily make the most happy and productive groups (“nobody here but us chickens”).

I have to quote this in full – see if it rings a bell for you: “Such a culture would place a greater emphasis on the derivative and collaborative nature of scientific advances, along with a reduced emphasis on rewarding hypercompetitive behavior and the cult of the “rock star” investigator. The evolutionary biologist David Sloan Wilson has recounted an attempt to improve the productivity of egg-laying hens by Purdue researcher William Muir. The approach of selecting the most productive individual hens from each group to breed the next generation was compared with selecting the most productive groups of hens. Unexpectedly, the latter approach was most successful because the individuals within successful groups had learned to function cooperatively, and the happier hens laid more eggs. Productivity plummeted when the star performers were grouped together, and all but three hens in this group were dead by the end of the experiment. After a lecture describing these results, a professor in the audience exclaimed, ‘That describes my department! I have names for those three chickens!’ In fact, we all know who those chickens are.”

7. A vision of a healthier scientific culture. Generosity and cooperation should be recognised and rewarded. “Science functions best when scientists are motivated by the joy of discovery and a desire to improve society rather than by wealth, recognition, and professional standing.”

8. How to build a motivated research community.
Efforts to reform science need to focus on developing supportive and productive groups:
“Individuals need to be matched with projects appropriate for their talents and passions and that they require both autonomy and connectedness with other members of the group. … A healthy scientific environment is one in which the freedom to do what one wants is complemented by support and stimulation from a community. This will enhance productivity and innovation.”

I had a sense of recognition in reading the article. I think these are serious issues which have grown up in the last 50 years or so (i.e., on our watch) so we have a responsibility to fix them and make science work better for the next generation. I also agreed with many of the recommendations, but I wonder how you feel. I’ve set up a google survey where you can indicate your view about each point – it shouldn’t take more than 5 minutes and the results, I hope will make interesting reading if we can get enough people to participate.

Update 16/4/12: original paper now featured by the New York Times.


Author: tomhartley

Neuroscientist and University Lecturer in Psychology

6 thoughts on “Hidden Gem III: What’s Wrong with Science?”

  1. I agree with most of this, but I’m especially concerned about the pressure scientists are under to publish. Like all scientists, I must read an endless stream of articles in order to situate my own research within that of the wider scientific community. Perhaps one time in twenty do I come away from an article feeling truly enlightened. This is a problem. I am not of the opinion that all data are helpful; I do not believe that every study should necessarily be considered another brick in the wall. This is not to say that the majority of papers are meritless on an individual level, they are not (and most display impressive examples of experimental ingenuity), but that their enlightenment-to-data ratio is too small to be helpful and can simply end up diluting the pool of work from which we must all hunt for relevant research. Perhaps this situation would be mitigated somewhat, if we had a means of properly indexing and standardising data across studies – then all data could truly contribute – but, in my field at least, this is not the case and nor do we have a natural language super-computer that can integrate all of the information that is currently hidden between idiosyncratic preambles and post-hoc explanations. I do not blame individual scientists for any of this. We must all do what we can to balance the need to publish and earn a living with the desire to produce original, truly enlightening work. Rather, it is the corrosive currency of citations that has forced the hands of well-meaning individuals and removed the natural filter of a scientist’s desire to publish at the point when they believe that they truly have something worth telling the world about.

  2. Great article. I think you raise some important issues, especially around the publishing status quo. It’s hard to unravel, but it’s clear that some of these problems, like the way the Matthew Effect has developed, are unintended consequences of things you’d think would promote excellence and public engagement, like making scientists compete for funds, and encouraging media coverage.

    I like the idea of the survey, but found the questions so loaded I couldn’t answer them with a straight face. I mean, I hope no-one responds “Yes, but it’s not a problem” to assertions like “Targets and competition are distorting science” or “Winner takes all competition encourages cheating and fraud.”

    Thanks for the read.

    1. @Matt You have a point, but a few people have responded “Yes, but it’s not a problem” to those statements. I can’t speak for the respondents I guess they are thinking that e.g., the distortion, cheating and fraud that are produce/encourage are insignificant (and might even be outweighed by benefits not mentioned in the article). Maybe I should have said “Yes, but it is not a significant problem”.

  3. Reblogged this on To the left of centre and commented:
    For a while wanted to write something about imbalances in the science career structure and in how people are rewarded in science. I have never quite been able to articulate it particularly well and so haven’t got around to posting something. The following is something I found very interesting and illustrates my general views and thoughts in a manner more coherent than I would probably have managed. Well worth a read. I’ve also never reblogged anything, so thought I should try. Hope I’ve done it properly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s