The Problem with Publications Rarely Publishing Dull Results

5
94

Medical, psychiatric, psychology and other scientific studies that simply replicate research, find null results, or get results showing no effects from treatments are far less likely to be published, and a New York Times op-ed looks at a few burgeoning solutions to these problems. Meanwhile, in an interview in Mètode, medical Nobel Prize-winner Randy Schekman discusses how the situation is distorting our collective understanding.

“(M)y colleagues and I propose a radically different publishing model,” writes Brendan Nyhan in the New York Times. “Ask journal editors and scientific peers to review study designs and analysis plans and commit to publish the results if the study is conducted and reported in a professional manner.”

Meanwhile, in Mètode, Shekman comments on various topics.

Shekman on the journals Nature, Cell and Science: “Yeah, they have a very effective business plan, I would say. They prey on people’s vanity.”

Shekman on news media and science: “In order to get their papers published in Nature or Science, they exaggerate the importance of their work, absolutely. Investigators are under pressure to get their work funded and so they think that if they have more publicity for their work, even in the newspapers, that will help them with funding, or help them with recognition, fame and glory. I mean, why should scientists be any different than other people?… But increasingly, unfortunately, these newspapers have fired, gotten rid of their science journalists. So now many newspapers are dependent on the press releases issued by the journals to describe their work, and that may be good for the journals, but I don’t think it’s good for the general public.”

Shekman on the lack of reproducibility of important studies: “We have a problem… But I think it is unknown what fraction of the literature is wrong…”

To Get More Out of Science, Show the Rejected Research (New York Times, September 18, 2014)

Randy Schekman (Mètode, Summer 2014)

5 COMMENTS

  1. “Medical, psychiatric, psychology and other scientific studies that simply replicate research, find null results, or get results showing no effects from treatments are far less likely to be published”
    Personally I consider this the top most important problem of science overall, especially biomedical. Actually people who manage to publish negative results tend to wear it as a badge of honour – it’s really tricky to do.

    Report comment

    • Any real scientist knows that negative results are far more important than positive results. Science is actually really bad a proving anything, because there are almost always multiple ways to explain a “positive” finding, as well as many ways to manipulate data to make a finding look more positive than it really is.

      The real test of science is trying hard to DISPROVE a hypothesis in any conceivable way and still having it hold up. Science is very good at absolutely disproving a hypothesis when the results contradict what is predicted to occur, so any events that contradict a given hypothesis are of the utmost importance, because their presence, or the lack of them after multiple efforts to replicate or disprove an idea, are really the bottom-line determinant of scientific truth.

      Null findings should systematically be published, and should be required to be submitted along with any “positive” findings before a drug is ever allowed to reach the market. If this were done, a lot of bad drugs would be stopped in the pipeline, instead of 10 years later after hundreds are killed and the patents finally expire.

      —– Steve

      Report comment

      • These are great points Steve, as usual but we can all see how in practice clinical trials don’t get published, data on side effects gets hidden, statistics are manipulated, patients groups are manipulated… I could go on. And even if there are negative results they pass unnoticed or get dismissed. Also as soon as the drug gets approved it almost never happens that FDA reverses its decision based on the data from the actual population. Unless 90% of patients die on the spot from the drug any additional data on horrible side effects and insignificant efficacy will be ignored or maybe warrant one or two lines in the side effects list.

        The first thing to do is to take the money of the drug approval process and make it 100% transparent. Pharma should stick to producing drugs not inventing them or even worse sponsoring or conducting trials. All trials should be registered before start, monitored, every patient should be followed (no kicking out patients from the dataset before they quit or died) and all results, including raw data should be published in full and open access. To me that sounds like common sense if patients’ health was what we care about but it’s apparent from the system that it is of minor importance in comparison to pharma’s bottomline.

        Report comment

          • Thanks for the recommendation but I’d prefer to stick to doing science rather than dealing with psychopaths on a daily basis. Unless there’s no one better to take that post – the you can’t count on me ;P.
            But seriously, my ideas are not rocket science or extreme agenda – they’re just common sense if you really care about the scientific validity and evidence based medicine and the fact that these proposals get dismissed and rejected by policy makers is proof that something else is more important here (a little clue: it may be money).

            Report comment

LEAVE A REPLY