It’s #overlyhonestmethods come to life!

Retraction Watch reports a study of microarray data sharing. The article, published in Clinical Chemistry, is itself behind a paywall despite trumpeting the virtues of open data. So straight to the Open Access Irony Award group at CiteULike it goes.

I was not surprised to learn that the rate of public deposition of data is low, nor that most deposited data ignores standards and much of it is low quality. What did catch my eye though, was a retraction notice for one of the articles from the study, in which the authors explain the reason for retraction.

The future of science publishing from 1996

Floating by in the Twitter stream, this from @leonidkruglyak. It leads to a light-hearted opinion(ated) piece by Sydney Brenner in Current Biology, 1996.

In 1996, you may recall, the Web was just a few years old. Amusingly (sadly?), it seems that Brenner predicted many of the topics in science publishing that we’re still discussing in 2013. It’s just that he thought they would be implemented in no time at all.

For example, open refereeing:

It is incidents such as this that have led me to question whether the anonymity of referees needs to be guarded so closely

Self-publishing/archiving and post-publication peer review:

The electronic pre-print with open discussion (not refereeing) will soon become commonplace; in fact, labs could go into the publication business by themselves

Demise of the journal impact factor, publishing economics and altmetrics:

We will need something to substitute for the present ratings given to papers appearing in ‘superior, peer-reviewed publications’ (and commercial publishers will find ways of making people pay for this)

Perhaps we should have a readership index; it should not be beyond the wit of man to devise a way of recording whenever a paper is read, hard-copied or cited

As Ethan said:

@neilfws what's taking so long?!

—
Ethan Perlstein (@eperlste) January 10, 2013

Reproducibility: releasing code is just part of the solution

This week in Retraction Watch: Hypertension retracts paper over data glitch.

The retraction notice describes the “data glitch” in question (bold emphasis added by me):

…the authors discovered an error in the code for analyzing the data. The National Health and Nutrition
Examination Survey (NHANES) medication data file had multiple observations per participant and
was merged incorrectly with the demographic and other data files. Consequently, the sample size was
twice as large as it should have been (24989 instead of 10198). Therefore, the corrected estimates of
the total number of US adults with hypertension, uncontrolled hypertension, and so on, are significantly
different and the percentages are slightly different.

Let’s leave aside the observation that 24989 is not 2 x 10198. I tweeted:

"an error in the code for analyzing the data" - retractionwatch.wordpress.com/2012/08/16/hyp…. Entirely avoidable if methods were published in full.

—
Neil Saunders (@neilfws) August 17, 2012

Not that simple though, is it? Read on for the Twitter discussion.

We really don’t care what statistical method you used

Update: as pointed out in the comments, the amusing error in this article has been “corrected” (or at least, “edited away”). Thanks for your interest.
Update: I note that this article is now “Highly Accessed” ;)

An integrative analysis of DNA methylation and RNA-Seq data for human heart, kidney and liver
BMC Systems Biology 2011, 5(Suppl 3):S4

(insert statistical method here). No, really.

With thanks to Simon J Greenhill and Dave Winter.

Fixed that for you

A comprehensive framework for prioritizing variants in exome sequencing studies of Mendelian diseases.

Why not also highlight the key genes by colouring cells in red?

You’re welcome.

When even your own publication list makes no sense

A few years ago, the head of my research group asked if I’d like to help write a chapter for a book. I weighed up the pros: it was an updated version of a previous book (so not too much work), it was invited (so not too many battles with reviewers) and it’s another item to go on the CV. The cons: typically, this kind of article appears in an obscure, closed publication that no-one ever reads or cites. So I said sure, why not and we wrote it.

It’s listed on my publications page at this blog as:

Saunders, N.F.W., Brinkworth, R.I., Kemp, B.E. and Kobe, B. (2010). Substrates of Cyclic Nucleotide-Dependent Protein Kinases. In: Handbook of Cell Signalling (Bradshaw, R.A., Dennis, E., eds.). Academic Press San Diego, 182:1489-1495. [DOI]

and sure enough, if you visit that DOI (and have a Science Direct subscription), you’ll find chapter 182 in the Handbook of Cell Signalling.

I thought no more about it, until I updated my Google Scholar citations page, where I found this:

Substrates of Cyclic Nucleotide-Dependent Protein Kinases
Neil FW Saunders, Ross I Brinkworth, Bruce E Kemp, Bostjan Kobe
2011/4/12
Transduction Mechanisms in Cellular Signaling: Cell Signaling Collection 399
Academic Press

And here’s the link at Google Books. Same article, same editors – but in chapter 41 of a different book: Transduction Mechanisms in Cellular Signaling: Cell Signaling Collection, on pages 399-405.

So apparently, my chapter has been “repurposed” for a completely different publication. Perhaps this transpired in consultation with the research group after I left. Perhaps there’s a long-forgotten email trail in which I agreed to this. Or perhaps we have so little control over our own work that strange things like this can just happen.

Reproducible research: three links that made me think

I’m constantly amazed, bemused and troubled by how little published scientific research is genuinely reproducible, in that you or I (or even the original authors) could go back and check the results. Three examples from around the Web converged in my mind this week.

Monitoring PubMed retractions: updates

PubMed cumulative retractions 1977-present

There’s been a recent flurry of interest in retractions. See for example: Scientific Retractions: A Growth Industry?; summarised also by GenomeWeb in Take That Back; articles in the WSJ and the Pharmalot blog; and academic articles in the Journal of Medical Ethics and Infection & Immunity.

Several of these sources cite data from my humble web application, PMRetract. So now seems like a good time to mention that:

• The application is still going strong and is updated regularly
• I’ve added a few enhancements to the UI; you can follow development at GitHub
• I’ve also added a long-overdue about page with some extra information, including the fact that I wrote it :)

Now I just need to fix up my Git repositories. Currently there’s one which pushes to GitHub and a second, with a copy of the Sinatra code for pushing to Heroku, which isn’t too smart.

Lists of URLs are so 1990s

Subtitle: “Why some projects are not worth your valuable time and skills.”

Let’s wrap up this exploration of how to extract URLs associated with NAR Database articles. I’m tempted to start with the summary: don’t bother – just Google it. If you want that, skip to the end.

Why can’t PubMed or academic journals get the basics right?

A recent question at BioStar asked “Is the NAR database list available in a computer readable format?” The short answer is “no” and Pierre has done some excellent preliminary work to address the issue.

I’ve been working on a database and web application to check the associated URLs but quite frankly, this is tedious, a waste of everyone’s time and could be entirely avoided if the publishing industry did a better job. All that’s required is that either NAR or PubMed provide structured data – XML, Medline format, I don’t care what – containing a field that looks something like this:

URL    http://a.valid.url.goes.here

That way, we could all avoid writing regular expressions to detect URLs in abstracts. No wait – to detect broken URLs in abstracts. You would not believe how many of them look like this:

URL    http://www.amaze.ulb. ac.be/
                            ^

Someone helpfully informed me via Twitter that this is “often a result of typesetting.” Thanks for that.