Reinventing the Biomedical Journal

Why do journals need reinventing?

Earlier today I dispatched to the publishers the corrected proofs of a book entitled “The Trouble with Medical Journals.” I wrote this book in a 15th century palazzo in Venice in February and March of 2003, after nearly a quarter of a century working as an editor on the British Medical Journals. I didn't expect to write a “J'accuse,” but that's how it turned out. Here is the last paragraph of the book:

“Medical journals have many problems and need reform. They are overinfluenced by the pharmaceutical industry, too fond of the mass media, and yet neglectful of patients. The research they contain is hard to interpret and prone to bias, and peer review, the process at the heart of journals and all of science, is deeply flawed. It's increasingly apparent that many of the studies journals contain are fraudulent, and yet the scientific community has not responded adequately to the problem of fraud. Editors themselves also misbehave. The authors of the studies in journals have often had little do with the work they are reporting and many have conflicts of interest that are not declared. And the whole business of medical journals is corrupt because owners are making money from restricting access to important research, most of it funded by public money. All this matters to everybody because medical journals have a strong influence on their healthcare and lives” (Smith, 2006).

The book produces evidence to back up all these assertions, but here I want to concentrate on reinvention. My point in reproducing the paragraph is simply to give some of the reasons why reinvention is needed. (It might also help flog a few copies of the book.)

Research studies appear on databases, not in journals

First, I don't think that it makes any sense to continue with paper copies of research articles. Instead of the “quasi-legal” document that is the current scientific article, we should be moving to full data being available on the web together with the software that might have been used to manipulate the data, as well as multimedia presentations to back up the data. Research papers are primarily of interest to other researchers in the same area, and they usually don't need the introduction and certainly not the discussion, which mostly degenerates to hype anyway (Horton, 1995).

Needless to say, I believe that these presentations should be available for free on PubMed Central or an equivalent. I can't see the point in assembling the research into journals. Researchers can easily be alerted, as many are now, to the appearance of new research relevant to them. Indeed, if all research appeared on one database it would be much easier to alert researchers. We wouldn't suffer as we do now from the research studies being “Balkanised” (split into many journals, many of them behind access controls).

The new peer review: learning from Wikipedia

Who would do the peer review, you might ask. My most radical answer is “the world at large.” Research presentations would simply be posted, as they are now in high energy physics and astronomy. The rest of the research community will then decide whether the research is high quality or not.

Peer review is, I believe, an empty gun. It's slow, expensive, largely a lottery, prone to bias and abuse, and poor to useless for spotting fraud and error (Jefferson et al., 2002; Godlee and Jefferson, 2003). In addition, it's anti-innovatory: come up with something really new and it probably won't get through peer review. After years of research into peer review its downside is supported by lots of evidence, whereas there is little evidence of its benefits. “If it was a drug,” says Drummond Rennie, deputy editor of JAMA, “it would never get onto the market.” I find it deeply ironic that something so central to science should be rooted in faith not evidence.

If an absence of peer review (or post-publication review, as I call it) is a step too far, then we should have an author (or rather funder) pays model. These fees could support a peer review mechanism, which should be open in that both authors and readers would know who was reviewing studies. It's ethically unacceptable that such important judgements should be made an unidentified judge. Like it or not, we live in a world where what is not transparent is deemed to be biased, corrupt, or incompetent until proved otherwise. Plus I believe that peer review should be a scientific discourse rather than an arbitrary judgment. This is far from radical: it's simply science returning to its roots when science was presented and discussed at meetings rather than published in journals.

Perhaps we will invent new forms of peer review by learning from innovations like Wikipedia. It is in some ways a form of peer review, only reviewers make changes directly rather than simply commenting. I'm clearly at the edge of my imaginings here, but that's the reality when a paradigm shifts: those embedded in the old paradigm find it hard (indeed, impossible) to imagine the new.

Academic credit in the new world

Another worry from the conservative about such a system is to wonder how credit would be allocated. At present credit comes from publishing in prestigious journals. Often the impact factor of the journal (a dubious and manipulated statistic) is allocated to the paper, which is wholly unscientific because there is little correlation between the citations to studies and the impact factor of the journals in which they are published, because the impact factor of a journal is driven by a small number of highly cited studies (Seglen, 1997). In the new world I'm imagining, credit would come from the buzz from researchers and hits on the study. These hits can be disclosed in real time, unlike citations, which come years after studies are published.

So what do journals do?

What would journals do in this new world? Most of them would die, which would be a great relief. But those that remain would do what journals do best anyway: encourage debate, inform, alert, discuss, disturb, set agendas, select out what matters, educate, and entertain. Rather than being passive carriers of research studies, half of which are never cited, they would be the engines that digest the important studies, incorporating them into the scientific body. These “new journals” would be much more fun than the present journals, and they would be much more interactive, with less distinction between authors and readers, lots of blogs and vlogs, and easy ways of responding to the material that they “publish.”

Cleaning up journals

Authorship will be gone in my reinvented world. Instead, we will have contributorship, where people simply describe what they contributed and a guarantor accepts full accountability (Renie et al., 1997). Disclosure of conflicts of interest will be mandatory, and any failure to disclose will be quickly highlighted in this very transparent world. Pharmaceutical companies will find it harder to get the results they want when they must post all data, including all adverse effects. The marketing departments of the companies will anyway be much less interested in complex presentations aimed at researchers rather than spun papers in leading journals, where they can buy a million reprints and use the lustre of the journal to promote their product. It may even be that research misconduct, fraud, will be more quickly rooted out. There were blogs highlighting the misdemeanours of Hwang Woo-suk long before anything appeared in Science.