A couple of years ago, I noted that some journals were not making the process of commenting on articles especially easy. My latest experience suggests that little has changed.
On April 26 this year, I read A comparison of feature selection and classification methods in DNA methylation studies using the Illumina Infinium platform. It’s an interesting comparative study, completely devoid of the code used to reach the conclusions. Nothing unusual about that – most published articles are similarly deficient. Still, I thought it was worth highlighting the issue. My comment:
This type of comparative study is potentially very useful. However, how are we supposed to reproduce the results or try the methods for ourselves without the code used by the authors? I find it baffling that journals publish statistical analyses without sufficient detail for others to reproduce.
Eleven weeks later…I receive a response:
Many thanks for writing to BMC Bioinformatics with a comment for one of our articles. We are in principle always willing to publish a variety of points of view. However, in this instance I should be most grateful if you could slightly rephrase your comments with a rather more positive slant.
We are an open access series and encourage authors to provide code and data to the readers of our BMC series journals.
Well OK…my comment could have been more constructive and I could have focused on the article, rather than drifting off into a more general rant about “the state of things.” And sure, site owners should have their own rules regarding comments – I do. That said: 11 weeks, to request a slight rewording of something not particularly offensive?
I’ll stick to discussing articles in near real-time on Twitter, thanks. Judging by comparison of my Twitter stream to journal websites, so will everyone else. Perhaps it’s time for journals to admit that in general, comments on the article page itself don’t work and just turn them off.
What You're Doing Is Rather Desperate 
Terrible reality. Once I left a comment in the Nature – it was a political news and I was saying that that is very pity that such a honorable journal went into dirty politics game. Soon after my comment was erased with words that it is offensive!
PLoS comments are actually really good, in-line with the text, and show the capacity of what comments could do, if more people adopted them. Which they might. So, I think this post is too nihilistic, unless you are trying to do it in the vein of “A Modest Proposal”, in which case, well played, but give us more clues!
Agree that the PLoS comment system is much better than the competition. Still, how many people comment at the articles as opposed to via Twitter? Answer: relatively, very few. What proportion of articles receive comments? Again, very few.
Thanks for the post, but I respectfully disagree. I think comment threads are the logical locales for post publication review. Please see the comment thread for my lab’s recent PLoS ONE paper here: http://www.plosone.org/article/comments/info%3Adoi%2F10.1371%2Fjournal.pone.0034024
I admit that most papers don’t have 14 comments; it didn’t happen by accident. I used a targeted email campaign to invite scientists with common interests to move discussions about my paper from emails to comment posts. And guess what? It paid off!
Comment threads are like gardens — nothing will grow without constant attention and a bit of fertilizer. I wish that journal article comment threads would blossom naturally but a commenting culture will take time to emerge given the academic cultural headwinds.
I don’t disagree with any of your points. It’s just that I find the implementation of comments at journals, in general, very disappointing. As someone said on Twitter: they need to turn moderation off and spam filtering on.
I think the implementation problem is with corresponding authors! They need to beat the pavement and solicit comments. I wish PLoS used the DISQUS commenting platform, but you know that old saying about bringing a horse to water…
At some point the frustration about the lack of commenting has to be directed at authors themselves. They have the power to cultivate discussion, but don’t seem to exercise it. Unless you can show me that authors made earnest attempts and were rebuffed, which is conceivable…
Dear Dr Saunders
Many thanks for your post – I’d like to introduce myself as Publisher for the BMC-series journals. I’m really sorry you had this poor experience with publishing a comment on BMC Bioinformatics. Having investigated, I’ve found we had a technical problem that meant the Executive Editor of the journal was not alerted to your comment for a considerable time. If you’d like to submit a new version, I’d be really happy to ensure it gets looked at straight away.
Further to that, you might be interested to know we are in the process of completely revamping our comments system as part of an overhaul of our technology. We hope and expect to be much more efficient in future! We aim to turn around comments within two working days normally, and should be faster still in future. Not quite as fast as Twitter, I know, but definitely better than 11 weeks :) And usually with fewer comments about what people had for lunch…
Thanks again,
Jo Appleford-Cook
Thanks Jo, I appreciate your taking the time to comment. I think a lot of publishers have got it all wrong with comments; no-one wants to see “YouTube” comment threads but on the other hand, I don’t see why perfectly legitimate comments should not appear almost immediately. As Lars says, I think the community has taken commenting into its own hands regardless of journals.
I think you are spot on Neil. The journals seem to forget that by commenting on their articles, we are doing them a favor. Waiting 11 weeks before answering that they will not accept your comment is simply unacceptable. In fact, not accepting a comment like yours is already problematic in my book; I cannot help but thing that what they find problematic is that you are in a way criticizing the journal more than the authors.
As a blogging the options seem pretty obvious. Either you can comment on a paper on your own blog, get it published immediately, with no censorship, and get full credit for it by creating content on your blog or twitter stream. Alternatively, you can submit a comment on the publisher’s website, with for a long time, risk having the comment rejected, and help produce content for the publisher rather than yourself. I know which option I would choose.
Ideally journals would have a trackback system that links to comments on Twitter, blogs etc. that are taking place elsewhere. Then they might not need their own comments.
Indeed and this is something that PLoS do quite well (each article having a Twitter stream). Or they might just use a more generic system such as Disqus.
Publications that require one to sign up for an account to comment, and thus add yet another username and password to the list one needs to keep track of (and add one more publication to the potential sources of marketing spam) is yet another nuisance of the commenting procedure.
Neil, I completely agree that commenting on journal websites is underused and often not a positive experience. Looking at the April 2012 dataset of PLoS Article-Level Metrics (downloaded from http://www.plosone.org/static/almInfo.action) I found comments, notes and ratings for only 13% of PLoS papers.
Two separate issues were mentioned in the comments: a) many authors seem reluctant to engage in comments to their papers and b) commenting on journal websites often has a bad user interface. As a technology person I think b) has to come before a). A good commenting system should make it easy for users to sign in and get approval for the comment, should integrate comments from other places (in particular Twitter), and should not focus on the article but rather the user – as a common comment stream for all journal articles. In an ideal world this comment stream would also contain other events, e.g. when someone bookmarks or cites the paper. No coincidence that this sounds a lot like FriendFeed. Disqus was mentioned, but I think I would prefer a better integration with the journal website.
Disclaimer: I’m technical lead for the PLoS Article-Level Metrics project.
I don’t think using Twitter for paper commenting is a good idea. There are issues with archiving Twitter, and it is not open-access. The logical place is to have the comments with the paper itself. And this means we should spend our efforts writing more comments, and writing to editors about the commenting system if it is poor. What LICENSE are my comments released under if I comment at PLoS or BMC ?
Perhaps “discussion” is a better word than “commenting” with respect to Twitter. Agree that Twitter has issues (and is in no way an archiving solution), but at least there is discussion there. Journals, it seems to me in general, don’t really care about their commenting systems, which is why I question whether they should bother at all.
“they need to turn moderation off and spam filtering on.” – Neil, then why does your blog say “Your comment is waiting for approval.” when I submit ;-)
My blog, my rules. I take your point :) Of course I don’t mean that moderation should be off, rather that it should ideally be so seamless as to appear to be off. You’ll note I rarely take more than a day to moderate here.
In case you haven’t seen it, Ethan today has posted a guest blog post about comments for the PLoS ONE blog: http://blogs.plos.org/everyone/2012/07/23/no-comment/.
Torsten, there are many things I don’t like about Twitter, but that is what most people currently use. More than 40% of new PLoS papers were mentioned in tweets since PLoS started tracking tweets a few weeks ago (a blog post on this is coming up). Now if only more tweets about papers had something interesting to say rather than simply retweet the title…
The PLoS guidelines for comments are at http://www.plosone.org/static/commentGuidelines.action, but I would have to find out about copyright on comments.
Pingback: What Users do with PloS ONE Papers | Gobbledygook