Too many trees out there
Science is broken: how can we fix it?
20 January 2013
The idea of eventual self-correction of inaccurate, wrong or manipulated data – that is the invalidation of false data by later correct papers – is in my opinion a myth
I remember being in some woods and thinking: “Damn, I’ll never come out of this”. There were just too many trees. I wasn’t alone, though: several undergraduates were as lost as me. I remember the Professor of Botany saying that in any forest there would be a number of plants that we wouldn’t be able to name properly. He used to stress that to come out of that situation we needed to see the picture as a whole, to identify the plants we knew among those we didn’t. “There are a lot of trees out there” was his favorite line.
That experience came back to me several times since the start of my career in science, in thinking about the crowd of postgraduates and post-docs around me, and the huge amount of scientific facts, figures and literature we were continuously producing.
Our job in science is extremely competitive. From our mid-twenties to early forties (the approximate time it takes to get from graduation to an academic permanent position) we are confronted with precariousness: temporary contracts, lousy money, and fierce international competition. Plus the peculiarity of the job itself: scientific experiments always have unpredictable output – if you can count on anything, it’s that what you get will be either uninteresting or unnecessary. Troubleshooting also takes a considerable amount of time. Hence, a researcher can work for several months – years – without producing publishable results. This is independent from one’s efforts or skills, but is rather intertwined with the nature of the project. And as years go by, the opportunities of switching into another career generally decrease, either because the bias against older applicants, or because people are unwilling to leave science, after having put so much time and effort into it – and because it’s a fascinating job that many people are loathe to give up, despite all its disadvantages.
A career in academic science is thus often a cul-de-sac. Of course, not all people with a degree wish to stay there. Many venture over to the “Dark Side” (what scientists picturesquely call doing research for a company); others opt for a parallel side track and take up science-related positions (e.g. in science support roles or education); some start a PhD out of indecision and just find themselves carrying on. Still, if you consider the multitude of people working at the bench in comparison to the few available permanent positions, you can easily feel lost in the woods. There are definitely a lot of trees out there, and in an overgrown forest there are two solutions: you either move the fence, or let the axe fall.
The sheer number of scientific papers we have to grapple with is the second hurdle. You can easily read ten to twenty hours a week simply to keep yourself up to date. Today, formal publishing is the major way to present, divulge and discuss advancements in basic research; also, it is the measurable output of a scientist. Lots of people competing in a limited niche where only publications count means that you need to publish a lot to be better than the others. But as straightforward as this may seem, the logic is flawed.
Generally, to get a PhD (and good academic employment afterwards), one to two first-author publications are highly recommended, while a double figure is prerequisite to getting a permanent position. That means publishing around one paper a year, which in the natural sciences is not trivial. And if you do not achieve that, well...where did you put that axe, mate?
The axe-solution to the overgrown forest – “publish or perish” – is widely distributed in science, and sometimes perceived as a good thing that enhances competitiveness and productivity. The scarecrow of external competitors is an effective deterrent against lazing around. In the current publication system, which appoints priority as a major good, people not only have to publish a lot, but have to present new stuff. The scientific value of a paper showing known results is approaching zero: if someone manages to publish the results you are striving for before you do, the significance of your work is eclipsed. Competition can be pretty heavy even inside a research group, too. Occasionally PhD students are hired on a temporary basis and assigned to the same project simultaneously, so that after a trial period, only the candidate with the most promising results is employed. Sometimes one-third positions are appointed, of course in reference to remuneration, not working schedule (50% positions are now the rule in natural sciences). Not happy with that? There are thousands of people willing to take your job over, sweetheart. You simply have to work more, work harder, work faster . Sixty-hour weeks or more are a reality in many highly rated research labs.
There’s a German word describing people only focussed on work who are experts in that field, but totally incompetent in others: Fachidiot. It comes from Fach (which means profession, compartment) and Idiot (guess that). The way things are now, we are in real danger of
This just doesn’t make sense to me. All right, there are limited resources (e.g. funding money); permanent positions cannot be filled with non-deserving scientists; a sane competition may vouch for high quality. But all this is easily pushed to the limits without considering the other side of the coin: too much pressure is often listed as factor leading to scientific misconduct; it can also cause sloppiness leading to false knowledge, promote Fachidiocy, and in the long run, is detrimental to the scientific community.
We scientists are idealists: riches and fame are not our goal. No, we’d like to do interesting stuff, discover something new, contribute. In the future, our time will be considered the Era of Science, and it’s thrilling to be part of it. Just consider the deep knowledge we possess, with the life-quality improvements and the practical applications deriving from it (technology, without science, is nuts). Still, the question remains: do we need knowledge to be produced at such a rapid pace?
If you answer with a wholehearted “yes”, that’s it: thanks for reading up ‘til this point, and hasta la vista. If instead you are a scientist stuck in a project which is starting to stink, are struggling toward a permanent position and feeling poorly rewarded, or haven’t forgotten yet your efforts to get there: stay tuned for a while. Now, I think there are basically two ways to introduce some positive changes: (1) the importance of reaching out to the wider world outside of science should be recognized and (2) the race toward publishing should be slowed down.
Everyone in science has experienced the frustration of not being understood by his family or circle of acquaintances. People ignore what our job is about and have in mind a stereotyped image of the scientist fuelled by science fiction. Science is something abstract for most people. For example, few non-scientists are able to name the two milestones in modern molecular biology, that is the discovery of the structure of DNA in 1953 and the development of PCR in 1983 ; fewer are able to name Watson, Crick, Wilkins or Mullis, all Nobel laureates (and even fewer can name Rosalind Franklin, but that’s another story ). I bet only a handful can explain the concepts lying underneath these findings. Those achievements are something so omnipresent in molecular biology and so many other fields of natural science that I always feel a sense of alienation when I realize people ignore them. I guess it must be close to what a tailor must feel if her husband only vaguely knew what dresses were.
There should be more effort to communicate the importance of science. With a significant fraction of scientific projects being in part or entirely publicly funded, the scientific community should feel the urge to communicate what our job is like, what science is about, and why it is so important for everyone (and so expensive). Nobody can do that more effectively than scientists themselves, and they must dedicate part of their time to it. We really need science outreach and a better recognition of its importance. To incentivize this, science communication must be acknowledged and counted towards a scientist’s outputs, along with his publication record. If we don’t want science to be uncoupled from society, better integration would represent a scientific advancement.
My second point is related: generating knowledge too fast is disadvantageous. In the current publishing system data presented in papers are not verified. True, papers undergo the meticulous scrutiny of peer-reviewing and are comprehensively discussed. But the truthfulness of data is not brought into question. The data are accepted – unless an evident misconduct is evident – and their significance discussed and criticized. Stunningly so, there is no group, authority or consortium which is devoted to verifying experimental data in papers. The scientific publishing system is based on trust, even if filtered through the eyes of criticism.
The idea of eventual self-correction of inaccurate, wrong or manipulated data – that is the invalidation of false data by later correct papers – is in my opinion a myth. There is a Catalan proverb that says: “When it floods, the first thing you run out of is drinking water”. If you want to discern right from false data, you don’t generate more, but instead you take a step back and try to find out the cause of the flood. Even without deliberate mischief from the authors, wrong data might still be present in scientific publications, partly boosted by the urge to publish too much too fast.
In a system in which knowledge is constantly piled up on pre-existing data, it is valuable to take a closer look at the published material out there. Priority is overestimated and the importance of verification of data should be appreciated. If the community were only more interested in publishing studies that validated previous papers, instead of only those that show something new, this would lead to independent affirmation. Even if such repeat papers were given less value than original papers, they would still “count”. As a nice side effect, you’d also reduce the pressure on scientists at the same time.
What about the issue of limited resources? My two solutions could help out indirectly there too. Science communication is less costly than experimentation; reducing competition and thus rivalry may lead to more cooperative efforts, and minimizing the number of people working on the exact same problem; and of course a better public understanding of science will result in a greater social acceptance and more public funding.
The days of my Botany course are long past and I came out of that wood intact. Still, I think often about my professor’s favourite saying. Maybe there’s no way out of it: to have a large messy pool of data might be the only way to fish out relevant stuff; science might only advance if based on merit and thus if pressure is put on its members. I wish nonetheless that we could practice our passion for science with more calm, and with more time to share our results with wider society. And that these changes would happen from the bottom-up, not mandated, but simply through a natural impulse stemming from scientists themselves.
We should stop saying that in an overgrown forest there are too many trees, instead of not enough room; if we favour struggling for limited resources, we should see that we run the risk of uncoupling science from society, and of losing the importance of science in murky waters. The drawbacks of exacerbated competition and the flood of scientific literature need to be acknowledged and rationally addressed: we cannot refuse to see the forest, claiming that there are too many trees out there.
 Amazing examples of real letters written by irate lab heads to group members they perceived as being slackers
 If you are keen to know what is behind the acronym PCR or want to read an exciting lab lit novel, I can highly recommend The Bourbaki Gambit by Carl Djerassi.