Responding to the Evergreen "research" article in Information Technology and Libraries

Update 2010-09-28: Fixed link

---

The home page for ***Information Technology and Libraries*** states:

*Information Technology and Libraries* ( ITAL) (ISSN 0730-9295) is a refereed journal published quarterly by the Library and Information Technology Association (LITA), a division of the American Library Association.

The September 2010 issue of ITAL contained an article by Sharon Q. Yang and Melissa A. Hofmann called "The Next Generation Library Catalog: A Comparative Study of the OPACs of Koha, Evergreen, and Voyager". As an Evergreen developer, I wonder just how much refereeing happened before this article was published. Certainly I am biased, but there are a number of problems with the study from my perspective:

1. The article stated "The latest releases at the time of the study was Koha 3.0, Evergreen 2.0, WebVoyage 7.1." Grammatical problems with that sentence aside, the first alpha release of Evergreen 2.0 was created on August 23, 2010. For an article published in September 2010, I find it highly unlikely that the authors were able to find any running instances of this version of Evergreen on which to base their information. Which leads to a problem with the methodology:

2. The stated methodology in the article was

   The OPACs used in this study included three examples from each system. They may have been product demos and live catalogs randomly chosen from the user list on the product websites. ... In case of discrepancies between product descriptions and reality, we gave precedence to reality over claims. In other words, even if the product documentation lists and describes a feature, this study does not include it if the feature is not in action either in the demo or live catalogs.

   This sounds like a thorough, pragmatic approach. But the product versions associated with each of the chosen examples are not listed. So while the article mentions the latest releases of each product, the actual reported experience might be based on an outdated version of the product. In the case of Evergreen, one of the chosen examples is two major versions behind the actual current stable release of 1.6.1.2, and another of the chosen examples is one major version behind the current stable release. In addition, one of the desired features of modern OPACs is customizability: not just the ability to turn features on and off, but also the ability to change the user experience significantly as a small matter of programming. Depending on which example OPACs were chosen for each system, the features the authors were looking for might not have been turned on or exposed.

3. On the "Single Point of Entry for All Library Information" feature, the authors state:

   While WebVoyage and Evergreen only display journal-holdings information in their OPACs, Koha links journal titles from its catalog to ProQuest’s Serials Solutions, thus leading users to full- text journals in the electronic databases.

   As far as I can tell, however, this is not a special integration feature of Koha; it appears to just be the use of an 856 with a URL that points to a link resolver for a lookup of a given ISSN. While it is a reasonable cataloguing practice, any other library system should be capable of that; Evergreen certainly is. However, check out this link for an example of how one can make an OPAC work harder by bringing resolver results right into the OPAC display. I built an Evergreen service, called open-ils.resolver, for caching resolver requests for ISSNs and used that service as the basis of a developer tutorial for writing Evergreen services. The idea isn't new; Jonathan Rochkind wrote about doing this back in 2007. But having a caching server-side implementation freely available for your library system is relatively novel. We've been using it since the summer of 2009. If you use Evergreen, then you can add this feature to your system too; it is written up in the developer workshop and is licensed under the GPL v2 or later, but if there's interest I can add it to Evergreen's core.

4. On "Enriched Content", the authors found that Evergreen offered only cover art. Of course, enriched content depends heavily on the content supplier and the chosen item. Since launching in 2006, Evergreen has provided enriched content such as cover art, abstracts, author notes, reviews, and tables of contents from Syndetic (requiring a Syndetic subscription, of course). In addition, Evergreen has offered Google Books integration in the form of partial & full previews (if available) inline in the detail page since Evergreen 1.6.0.0, thanks to the initial efforts of Alexander O'Neill at the University of Prince Edward Island. And as of Evergreen 2.0, the default content provider for cover art and tables of content will be OpenLibrary. Here's an example from our catalogue that brings in enriched content including cover art and a book review from Syndetic and a Google Preview.

5. On "RSS Feeds", the authors make the bold statement: "Koha provides RSS feeds, while Evergreen and WebVoyage do not". In the case of Evergreen, that's a laughable statement, because significant parts of the OPAC are built on RSS feeds. For example, in any Evergreen system, click on the "Basic catalogue" link and you'll find that it is nothing more than an RSS feed with a simple search form. If you are using Internet Explorer or Firefox on an Evergreen site, you might notice the search source selector widget is highlighted; that's because Evergreen is an OpenSearch provider, so you can easily add an Evergreen site to your browser as a search source. The OpenSearch results, of course, are built on RSS/Atom just like the examples in the OpenSearch description document. The default format that a user's custom bookbags are exposed in is also an RSS feed. I suppose the authors didn't find an RSS feed icon lighting up in the search results in the dynamic Evergreen OPAC and made the assumption that no RSS feeds were provided. To address this gap, I have added the one line of JavaScript to the default OPAC skin that adds the Atom feed link necessary to make the RSS feed icon light up. Not that many humans actually use RSS feeds directly - but it will help make it easier to find for future feature comparison articles.

6. On "Relevancy", while Evergreen does not currently use circulation data or "popularity" to affect relevancy rankings, I would happily argue that the out-of-the-box relevancy ranking algorithm is good enough to keep relevancy as the default sort option, while the relevancy algorithm of our previous ILS was simply terrible. Combine that with your ability to customize the relevancy algorithm, and I think an argument could be made that, while "Relevancy has not worked well in OPACs", it works well in this one.

As the article mentioned the latest release of Evergreen was 2.0, let me show you a screenshot of the default OPAC in Evergreen 2.0 as of the upcoming alpha2 release. Notice a few things:

• Those facets on the left are much closer to what the world has come to expect from a faceting interface. You get a narrowing effect on your current search results, rather than firing off a brand new search. There are pros and cons to this, but oh well.
• Notice that the RSS feed icon is lit up in the URL bar. Yes, Virginia, Evergreen has RSS feeds for search results, amongst many other things.
• The inline advanced search interface shows that Evergreen 2.0 offers an OR option, and clearly labels the relationships between the search terms.
• The OpenSearch source has been added to the list of Firefox search sources in the top right box, just by clicking on the icon and selecting "Add Evergreen catalogue"