An all for nought day

You know those days when you spend hours investigating various software packages – download, unpack, compile, tweak, test, discard – and at the end of the day, none of them worked as you hoped and you’ve achieved essentially nothing? Today was one of those days.

My quest for a usable Linux bibliographic solution continues. Here’s what I want:

  • Automated searches of PubMed with a set of topics at set intervals
  • Storage of results either in MySQL or a standard ASCII format
  • Search facility
  • Citation of selected references preferably into an OpenOffice document
  • Optional: multi-user, shared and personal databases

I’d had my hopes pinned on RefDB, but it has become very broken, at least on my system. I had it working a year or two ago but the latest version won’t even compile. It requires the latest libdbi (0.8.1) – Debian unstable is 0.7.2 and all my attempts to grab the latter version and compile against it have failed. It looks to me like a project suffering from over-engineering, always being tweaked to the point of unusability. A shame, because it has good import/export facility – it would be easy to run a weekly search via cron and Perl code using Bio::Biblio, import the pubmed xml direct to MySQL and export as RIS or whatever.
I tried Refbase which looks quite nice but lacks import filters. So now I’m thinking of a solution that gets PubMed xml in an automated search, runs it through Bibutils for conversion and just stores as flat files. Pybliographer would feature in there somewhere as might Bibus.

8 thoughts on “An all for nought day

  1. The refbase version that’s available from the SourceForge CVS supports import of records from various common bibliographic formats (Endnote, RIS, BibTeX, MODS) and web databases (MEDLINE, PubMed XML, Cambridge Scientific Abstracts, ISI Web of Science, Copac).

    Upgrading refbase v0.8.0 to the most recent CVS version shouldn’t be too hard:

    Automated searching of online databases is currently not supported but sounds like a very useful feature which should be investigated…


  2. Great. I will give that a try, as I do like the look of refbase.

    update: OK, CVS installed (manually), with some fiddling and editing. PubMed XML import works fine. Current drawback is that you have to paste into text box rather than uploading file. My CVS also lacks the contrib dir with the command line Endnote import script? Other command line scripts would also be good for combined search/import using cron.
    I like the look of this package, we’ll see how it develops.

  3. Probably a little late, but … I’ve been using Bibus for my thesis and so far it seems decent. It should support all the things you are asking, except maybe the multiuser support. The OpenOffice integration can be a little clunky sometimes, but the most recent versions have worked flawlessly for insertion of citations and bibliography formatting. (Now if only I can get around to writing the interface to CiteULike, it will really start to rock … )

  4. Never too late here. Great that Bibus is working for you, it definitely seems promising.

  5. Neil,

    inspired by your post, I’ve written a Perl command line client for refbase which allows you to search a refbase online database from the command line and retrieve results in various citation formats (HTML, RTF, PDF, LaTeX, structured or plain text) or export formats (BibTeX, Endnote, RIS, XML).

    You can try the command line client e.g. with this refbase installation:

    I understand that you’re mostly interested in CLI interfaces that will actually *add* data to the database. I have plans to develop another CLI client that will allow to batch import records from supported formats via the command line.


  6. Excellent. I’ve been away from reference management for a while, but will be sure to give this a try.

  7. Quick followup (in case anyone is interested): I’ve completed another refbase CLI tool which enables you to upload data in various formats to a refbase online database from the command line:

    This allows to integrate refbase with a custom workflow. For example, you could do stuff like automatically fetching & importing newly added records from PubMed via a scheduled cron job. The tool can be tested with

    Hope this helps,

Comments are closed.