IntroductionIn 2004 the Belgian Healthcare Knowledge Centre (KCE) made the choice of EndNote as its reference manager software among a list of 6 candidates. Since then, the software has been updated twice up to the EndNote X4 version.
Today, we are considering an update of our software but as a public institution depending on public funding, the KCE has to take part of a global effort to reduce costs in public sector. Among other actions, the move to open source software is considered whenever possible. As an important tool in the publication process, the reference software should not be replaced without a thoroughful evaluation. Other aspects than licence cost must be considered, such as features and training. The presentation will describe the steps in this evaluation process, with an emphasis on the retained evaluation criteria in order to answer the following question “Is Zotero or Mendeley a good alternative to EndNote for our team?”.
In order to evaluate possible candidates rigorously and with an
Evidence-Based mindset, we designed a methodology based on an analytic
hierarchy process. To do so we defined a set of features of the
different aspects perfect reference management software should have to
meet our institution needs.
Features were grouped in higher categories: user interface, collaboration, import and export, publishing, performances and scalability, security, documentation and budget. A weight was assigned to each feature in order to compute a score for each software and to choose the one with the highest score as our new reference management software. Some of the features were considered as critical and all candidates must comply with our needs for these features. These critical features are the handling of big databases (at least 3000 references) and the ability to design custom import and export filters.
The evaluation procedure we decided to follow has been shorter than we expected. Indeed, the main competitors to EndNote X7 failed quickly due to weaknesses on features we consider as critical: performances and custom importation filters. Endnote X7 is for now the reference management software which, even if not perfect, fulfil our needs at best. As the software in this domain quickly evolve we keep looking at the upgrades and new software coming on the market.