Can I Just Say…?

This week psychologists, neuroscientists and statisticians have been again prompted to reassess our methods by a widely-circulated preprint (pdf) accusing some in the field of acting like “methodological terrorists”. Thankfully, the initial urge to identify with one or other “side” (and to hit back at “opponents”) has largely given way to more thoughtful, nuanced and long overdue discussions about how research should be conducted. I am not arguing for or against one of the “sides” in this debate, but I am just sharing my perspective.

Scientists have learned that we need to continually question our beliefs about the natural world, and test them against the evidence. As Feynman put it: “The first principle is that you must not fool yourself – and you are the easiest person to fool.” We have to question our beliefs. This applies to our beliefs about the scientific method as well as to our beliefs about the natural world. It applies to ourselves as well as the other people who we think are wrong.

Continue reading “Can I Just Say…?”

Brash or brilliant?

How do we select people for career advancement, and are these processes really accomplishing what they set out to achieve? In these posts I am mainly concerned about my own field, science and academia. My worry is that the best scientists are not always selected, and that some groups, such as women, are systematically disadvantaged by our current processes. I think this is a problem, not just because of the unfairness it entails, but also because it means that the quality of the whole enterprise is being undermined. Although I am thinking principally about science and especially about women in science, I think the issues are much more general and so where possible I am going to try to write about them in general terms.


Ampersand Cartoon by Barry Deutsch. Thanks to Rachel Jones for sharing it on Twitter

As I plan to write a few posts on this topic, I want to begin by dividing the processes I refer to into two, potentially separable, parts: criteria and evaluation. By criteria I mean the objective characteristics we use to determine whether someone is doing well at their job. By evaluation I mean the (often subjective) way we determine the extent to which someone meets these criteria. Flaws in the criteria and evaluation would imply that the people best able to do a job are not always selected for advancement. Biases in the criteria and their evaluation would mean that these errors will work systematically in favour of some groups and against others. Neither flaws nor biases need result from a conscious intent on the part of those involved. Flaws might be due to unavoidable gaps or noise in objective information, leading to uncertain measures of objective criteria, and in the case of bias there is ample evidence to demonstrate that unconscious factors play an important role in human judgements under uncertainty. If we agree that our procedures might be flawed or biased, we should identify and fix these problems regardless of whether they are intended or unintended. In further posts I want to return to the issue of bias and how bias plays out in selection procedures and over time through career advancement. But for now I want to raise a problem that I think gets too little attention, and that is the question of whether our criteria are flawed or biased. I think they often are. One case identified in a blog post by Jenny Rohn looks at the effect of eligibility criteria relating to age and experience for a particular fellowship scheme. In this example, she explains how a small and apparently innocuous change in the wording can exclude candidates with atypical but promising paths involving career breaks, childcare responsibilities or industrial experience. The apparently reasonable aim of the criteria is to target those whose rapid progress demonstrates greatest potential. The unreasonable effect is to exclude those whose less rapid progress is unrelated to their potential. The general point is that, whenever we use criteria that do not directly correspond to candidate’s relevant abilities and achievements, we run the risk of introducing flaws that could lead to unintended bias.

It is clearly the case that when we select among candidates for a job or promotion we will want to choose people who are better than others or, in absolute terms, excellent by some yardstick. It goes, frankly, without saying. So why do job adverts so often specify that applicants will be “excellent”, “outstanding”, “world-leading”, “exceptional”, and so on? This can only have the effect of encouraging applications from people who think they’re fantastic and presumably deters more modest people, regardless of their objective abilities. In my view adverts, job specifications, funding eligibility and promotion criteria ought to focus on objective characteristics and scrupulously avoid terms that are open to subjective interpretation.

Wording of a current ad in This is purely illustrative of a widespread trend and no particular criticism is attached to the employer. After all, who wouldn't want to employ an extraordinarily gifted scientist.
Wording of a current ad in This is purely illustrative of a widespread trend and no particular criticism is attached to the employer. After all, who wouldn’t want to employ an extraordinarily gifted scientist.

Flawed criteria are particularly problematic because we tend to equate the outcome of a selective process with the desired result. For example, people will say, “as a modern scientist you do need to be competitive and confident – these are important characteristics” but this is rather circular. The reason a scientist needs to be competitive and confident is that only then will he or she win grant funding, publish numerous papers in glamour journals, gain promotion and win prizes, but this in turn is only because in each case evaluation criteria place undue emphasis on confidence and competitiveness. You do not need to be confident to do excellent science that will make an enduring contribution to knowledge. It seems very likely to me that in science the brash may at least occasionally outcompete the brilliant. The central point of this post is that if we want the best person for a job, we must ensure that our criteria for determining what’s “best” are aligned with what we actually want the person to do. Assuming we have such criteria, it’s still possible that subjective evaluations based on them will be affected by bias, and in future posts I want to show how the mechanisms of bias can be surprisingly subtle and long-lasting.

On identity: some people are scientists – deal with it!

Prompted by a workshop on social media that I am helping to run tomorrow, in yesterday’s post, I wanted to think about the role of social media in academia. But I went off on a bit of a tangent, remembering my early experiences with the web in the mid-90s. In those days, there was some resistance and scepticism about a technology that we now take for granted and use every day. Will people feel the same way about social media soon? Are worries about social media more justified? If there is a place for social media in academia, what is it? Can they bring benefits comparable to those we gained from email and the web, and if so, what are they?

I promised myself I’d answer all these questions in my next post, but as I began to write I found I had too much to say about the benefits, so I have decided to break it into installments. The first installment deals with an important use of social media in academia that arises from the way the internet has changed things. In the pre-internet world, ordinary people got to know one another by face to face meetings, by talking, by writing letters. Now more often our first encounter with a new person is on the net, either viewing a professional web page or a social media profile or perhaps through a medium like twitter. It seems important to me that I exert some control over these first impressions.

An important facet of this blog and my twitter account @tom_hartley, is that they allow me to be who I am. I am able to express a wider range of opinions and interests than I can through my scientific output, and to a wider and more diverse readership. (Also, I don’t have to worry about what a peer reviewer might say, which is enormously liberating.) In my view, this is important for me to maintain and project a sense of identity that extends beyond my particular area of research and teaching, even beyond my job into (gasp!) everyday life. Social media help me to establish a permanent presence on the internet where I can be somewhat independent of my job, and yet still engage with my work and the things that interest me at work and outside it:

OK, so exposing myself as a rockstar manqué, and shamelessly plugging this on my blog is a personal choice, and other people might feel more comfortable keeping it strictly professional. Either way I think an independent presence is becoming increasingly important for younger researchers who are likely to be employed on a succession of short-term contracts. By carefully maintaining their own area of the web, and their own profile on social media, they can project a good impression without being excessively dependent on their current employers.

This upside to social media comes hand-in-hand with one of their prominent downsides: it’s equally possible to make a bad impression. Step away from the keyboard when tired, drunk or angry. Consider maintaining pseudonymous accounts for the stuff you don’t want your employer or prospective employer to see. Pay close attention to your Facebook privacy settings. Don’t get drawn into hyperbole and personal attacks on others, however provoked. There is always someone who is being gratuitously wrong on the internet, so you can’t win. Hanlon’s Razor has been recommended to me: “never attribute to malice that which is adequately explained by stupidity”

As well as giving me more control over the way others see me, and I suppose there is a bit of vanity in this, I think it’s of more general importance that scientists see themselves and are seen as ordinary people with ordinary interests outside the lab. There should not be a disconnect between science and the rest of our culture, because science ought to influence and be influenced by the rest of society. I’ve always been interested in science and I like to see myself first and foremost as a scientist rather than for example, a psychologist or cognitive neuroscientist or a hippocampal or spatial cognition or memory researcher. The more specialist labels seem a bit more comfortable, perhaps because they come with the sense of security that in my little area I can make a reasonable fist of pretending that I know what I’m talking about. But knowing stuff isn’t what attracts me to science and I think the idea that science is just about existing knowledge is very misleading. For me it’s the stuff I don’t yet know that really excites me, and maybe I can convey that a little more clearly if I am prepared to step out of my comfort zone. So, although I felt a little uncomfortable identifying myself as a scientist at first, I started to feel that not identifying as a scientist was creating a barrier. When I took part in the excellent “I’m A Scientist Get Me Out of Here”. Finally I could say what I’d always felt. I’m a scientist. Some people are scientists – deal with it!

What’s Wrong with Science: Your Views

Last week’s post highlighted a paper by Casadevall and Feng which argued that the culture of science is in urgent need of reform. In particular, they argued, “incentives in the current system place scientists under tremendous stress, discourage cooperation, encourage poor scientific practices”. I agreed with the thrust of the article, and I wondered how widespread these views might be, so I set up a survey to poll readers on each point raised in the original article.

This slideshow requires JavaScript.

Continue reading “What’s Wrong with Science: Your Views”

Hidden Gem III: What’s Wrong with Science?

The final hidden gem is this article on reforming science. Although it appears in the journal Infection and Immunity it is concerned with the culture of science as a whole and will be of interest to all scientists. The authors Arturo Casadevall and Ferric Fang diagnose some serious systemic problems with 21st century science and prescribe some reforms by way of treatment. At the bottom of this article there is a survey for you to express your views.
Continue reading “Hidden Gem III: What’s Wrong with Science?”