Academia takes a very narrow view of what constitutes “output”. Rewards (such as funding or tenure) are given out on the basis of (1) publications, preferably first-author, preferably in so-called high-impact journals; (2) citations, in the same journals and (3) previous rewards – “demonstrated ability in securing funding”. I always find that last catch-22 clause particularly amusing.
I started to think about this when I read What is principal component analysis? [DOI 10.1038/nbt0308-303], in the current issue of Nature Biotechnology (subscription only). Now, I’m not criticising the article or its publication: it’s well-written, educational and a good basic overview of PCA for biologists who have not previously encountered the method. However, my first reaction was to recall a number of excellent blog posts on the same topic that I’ve read recently. For example:
- Deriving biological meaning from principal components analysis – by Mike
- PCA at Open Mind part 1, part 2, part 3, part 4
The Nature Biotechnology article is recognised by academia and qualifies for academic rewards. The blog posts – which are longer, more detailed, written by enthusiastic communicators and in theory, accessible to a much wider audience (as opposed to people with a subscription to Nature Biotechnology) – are not.
It doesn’t seem right to me. How does your institution evaluate and reward “non-traditional” output?
What You're Doing Is Rather Desperate 
In my host institution I would say that simply non-paper doesn’t count. Sometimes one can be recognized as a good science educator, but again it doesn’t count very much when climbing scientific carreer ladder.
The situation you’ve described is not fair (but it’s getting more popular – just to mention recent publications in PLoS CB). But what can we do? In all discussions I had on the open science I was getting the same argument over and over again – to rank scientists one need a peer reviewed metrics. And honestly the only reasonable thing that came to my mind was to escape the system (which I did), because I don’t see any significant change on a horizon in my home country.
I concur with Pavel’s opinion. I think at the same time it is difficult to measure the contribution made by a blog entry.
Also in my case non-paper contribution does not count. I don’t even think my supervisor reads mine, or my colleagues take advantage of what I try to write about Python.
You’d have to check with Andy Marshall at Nature Biotech, but the article to which you link is called a “Primer”, ie not a research paper, therefore would not “count” in the sense you mean — “academic rewards” are for peer-reviewed, citable research articles. Anecdotally, many scientists are often not respected by their peers for writing popular journal articles such as News and Views, etc. And they certainly don’t do it for the money.
One obvious difference between a journal and a blog article is that the former is independently edited, and has an external judgement applied to it to some degree or other. A blog post is usually written by the blogger concerned and has no independent scrutiny before it is published. I don’t know how many rounds of revision this author did before his Primer was published in N Biotech, but at Nature, authors put in several rounds of revision, plus check edited text, read proofs, often have figures generated for them by the journal. Also, of course, they are in a printed product — but I appreciate that bloggers don’t see print as anything other than a moribund medium ;-)
Thanks for the info Maxine – yes, there are articles other than peer-reviewed research in journals. However, I don’t think that alters my original post. I can tell you that academic researchers will still enthusiastically add articles like the NBT example to their CVs on the grounds that it was published by a respected journal. On the other hand they are unlikely to list their online activities – most likely because they don’t participate in any, but often because they feel that academia in general attaches no value to them.
Peer review always comes up in this debate. Like everyone else, I don’t have an answer to how web content can be validated. I will say – the argument that absence of academic peer review will lead to waves of unsubstantiated nonsense deluging the internet doesn’t wash with me. I think the web is more about how individuals filter the content that interests them, as opposed to filtering by anonymous experts, several of whom probably have a vested interest in denying publication of the content.
OK, I admit that I don’t have a lot of respect for traditional academic peer review :)
I guess the lesson is, in the current system where only peer-reviewed journal articles ‘count’, blogs can be a great way to test the waters with an article and get some writing practice. Treat the comments as pre-peer-review system, maybe invite some co-authors from that group, then convert your efforts into something that counts by cleaning it up and submitting it for publication in a peer-reviewed journal. Sort of like how Nature Precedings may operate.
Maybe one day we will have a Whuffie system for blogs, but until that day I’d say converting to ‘real’ publications is the best way to make the most of your blog, career wise.
I’m sure you’ve got some real dead-tree worthy material in amongst your archives Neil … maybe you could clean something up and submit it somewhere ?
I didn’t actually mention peer-review, Neil. I was talking about rounds of editing that a journal like Nature applies. I can’t speak for N Biotech but I imagine it is the same. At Nature, for example, we hire four full-time editors on our News and Views section (actually 3.5 as one of them has other duties too). Their job is to identify, commission and edit, edit, edit and edit about 10 journal pages a week. That is pretty intensive, no? Peer-review does not come into it. At Nature journals, the non-peer-reviewed parts of the journals have enormous editing resources put into them (News, Commentary, Book Reviews, Nature Jobs, Correspondence, Editorials, etc).
Apologies for omission in my comment: should have read
I didn’t actually mention peer-review *in my second paragraph*…. I was making the point that journals are different from blogs in their non-peer-reviewed material. The article has to be considered acceptable by an independent editor, then go through an intensive process that is not peer-review.
Another PS, naturally I differ from you in your assessement of the peer-review system. “I think the web is more about how individuals filter the content that interests them, as opposed to filtering by anonymous experts, several of whom probably have a vested interest in denying publication of the content.”
I think that unfiltered content (wherever it is published, web or book or wherever) is on average less accurate, more error-prone, less interesting to read than content that has had some filter applied to it. Your flip characterisation of peer review ignores (1) the editor and (2) the fact that several referees (all independent) of a paper see each others’ comments at revision/resubmission stages. Your comment about the “web being about filtering….” etc seems to ignore, for example, scientific and medical “misinformation” sites (of which there are many – eg medical cures, climate-related, creationism, etc), the absence of a managed correction system for errors, and the propensity on the web for lots of “recreational outrage” as I have seen it called. And so on.
I just think it is a straw man to put blogs “against” journals. They are different, and they can coexist quite happily. I’m a journal editor and a blogger, I enjoy reading content in journals and in blogs.
I suspect we are talking at cross-purposes here. Or perhaps I’ve allowed too many of my random thoughts to pollute the main thread.
All I wanted to say was: (1) there are many excellent contributions to science communication from what academia would call “non-traditional outlets” and (2) when we do our annual appraisals for career development purposes and list our contributions, it would be nice if the academic work environment recognised some of them as having value.
I’m not interested in pitting blogs against journals, debating the merits of the peer review process or the editorial standards of NPG publications (which I’m sure are excellent).
Pingback: Science in the open » Giving credit, filtering, and blogs versus traditional research papers
Pingback: Michael Nielsen » Biweekly links for 03/14/2008
Pingback: Michael Nielsen » Open science
Pingback: Publishable science
Pingback: Knowledge production pipeline « Monday Begins on Saturday