Correction update: 2007/06/26Wow. I am incredibly embarassed. Somehow, I made a very stupid mistake in my summary of the State Library
of Ohio ILS Options Discussion Meeting Minutes - April 24, 2007. The mistake was that I incorrectly attributed Joshua Ferraro of LibLime with making statements about Evergreen at that meeting when he was not even present. All of the statements about Evergreen should have been attributed to Stephen Hedges. I apologize profusely to Josh for this mistake, and will repeat this correction and apology in the section of the blog entry closer to the text.
I have been in the process of gathering information about the possible future
of our library system, with a focus on SirsiDynix's Rome, Evergreen, and Koha,
for a number of months now. This results in having to sift through claims from
a number of different sources about the capabilities (present and future) for
all of these systems. In the context of a recent post on the open-ils.org blog (Lies, Damned Lies, and Library Automation Software),
as well as all of the shilling that will undoubtedly be going on on the exhibit
floor and in the hospitality rooms of ALA, and finally building on
Karen Coombs' post on Bias, Objectivity and Authority, I would
like to make a point that ideally shouldn't need to be made (especially for
librarians!), but sadly seems to be necessary in the context of discussions about Evergreen and Koha.
The point? "Know your sources." And "Check your facts." When you've been given information about
something, you don't blindly accept the information as given - you check the
references and determine the authority of the source. This is par for the
course for reference librarians educating patrons performing research in their
libraries, but oddly enough seems to be a common blind spot when it comes to
performing evaluations of the software that powers your libraries.
Problem #1: Evaluating information about a given product from the company or
organization that stands to benefit from your adoption of that product.
This is especially hard when dealing with companies offering proprietary
products that won't hand you an evaluation copy to try out in your own
organization; or that run closed mailing lists; or that don't make their
documentation or support infrastructure openly available.
But it can also be hard with companies or organizations offering open source
products or support for open source products. The company or organization may
point you at an online demo of their product, but that demo may reflect a
heavily customized, bleeding-edge version of the product that mere mortals
cannot install - or even more insidiously, may include code that is not
currently included in the open source repository.
The good news on the open source front is that independent contributors have
made VMWare images of some of the most popular library systems available
download that reflect a standard install of the product directly from the open
Note that a common approach to marketing a product is to provide a feature
list - basically a checklist of features. A naive decision maker might assume
that more is better, which often results in products breaking down the
features that they do well into many sub-features. It's a form of checklist
inflation - but as long as you've got your eyes open, at least it's more
information rather than less. For each feature you're actually interested in,
you have to ask a couple of additional questions:
How many other libraries are currently using this feature? It may be
great that the software you're looking at includes LDAP authentication as an
option, but if there's only one other library using the feature it's
unfortunately likely that they will be using a different LDAP directory
product than you (Novell eDirectory vs. MS ActiveDirectory vs. OpenLDAP vs.
IBM Directory) and they will be using it in a different way.
Is this feature part of the base package, or is it an optional extra
that's going to cost me more? Not such a problem with the open source
options, although it depends in that case whether you're buying commercial
support for the product. The model that the company uses may be all-inclusive
or a menu of different costed support options.
Is this a massive feature that hasn't been broken down into
sub-features? The danger here is that, for the purposes of looking good in
feature comparisons, a product may have added a number of "features" that
really just scratch the surface of what comparable products offer.
For example, if a library systems product says it has a serials module and an
acquisitions module, you have to dig into what that really means. Does the
"serials module" just mean that it will spit out a routing list whenever you
check in a new issue of a given serial? Or does it mean that it handles
predictions, claiming, holdings, etc., in the way that meets your library
needs? By "acquisitions module", does the product mean that it simply records
the cost of each item that you have acquired? Does it allow you to make
on-order items visible in the catalogue, with the ability to place holds? Does
it include EDI capabilities? Does it provide a complete fiscal management
system with funds and reporting and electronic record import / export hooks
for the ERP system that your university or municipality uses so that costs and
invoices don't have to be manually entered multiple times in multiple systems?
Perhaps most importantly, does the system have the flexibility to adapt to your
needs, or does the system require you to adapt to its needs? Can you live with
the 80% of functionality that most sites need, or does your site live in the
long tail of requirements. In the case of serials, for example, do you need
the ability to specify any pattern, or can you just deal with irregular
patterns those as exceptions?
Problem #2: Evaluating information about a given product from a company or
organization that offers a competitive product.
Sales people make it their business to know their competitors so that they can
accomplish two goals:
Focus attention on (and often embellish) their own product's strengths,
and know how to spin responses to their own product's weaknesses that might
be identified by a competitor.
Identify the weaknesses of their competitor's product, particularly when
their own product has comparable strengths in that same area. Note that these
weaknesses don't necessarily have to be real, they just have to be believable
and hard to disprove.
Among the proprietary options, without having access to a hands-on test system,
the system documentation, or the product mailing lists, it is incredibly hard to
verify claims about a products' strengths or weaknesses. Even for claims about
the future development of a proprietary product that you already have access
to, the company cannot be held liable if plans change. Companies can, for
example, cancel an entire product even after beta versions of the product have
been released into the wild for "development partners." Horizon 8, anyone?
Development partners: test our beta for us!
Oh - on the topic of "development partners" - this is typically a euphemism for
"we'll give you a discount on product XXX if you put it into production and
report all the bugs you find." Companies love this approach because it gives
them visibility in the marketplace ("Look, we already have deployed product XXX
to five sites! It's proven and ready for you!") while enabling them to
effectively continue development on product XXX and hope to have a polished
product ready in time for the bulk of their potential customer base to actually
adopt it. In the past, Microsoft very effectively used the "product announce"
to prevent customers from purchasing a competitor's product that offered
compelling features and stalling the decision long enough to then develop and
bring their own product to market.
Disinformation and open source projects
Surprisingly, the disinformation approach works even in open source projects.
For example, I have read and heard claims about Evergreen like: "Oh, Evergreen
is just for massive consortiums / it needs 40 servers to run / it doesn't scale
down to just a single library." You can see how this could be believable if you
don't push too hard on the claims, because Evergreen's was developed for a
consortial library system and much has been made about the impressive server
cluster that GPLS runs Evergreen on -- however, having run Evergreen on a single
VMWare machine on my laptop, I can personally attest to its ability to scale
down to a single server (or portion thereof). And you can run that same VMWare
image on your own laptop or spare desktop machine and disprove that claim
yourself; but many of the decision makers do not have the technical skills,
time, or interest to get hands-on with products like Evergreen. So they have
to trust what they read or hear, hopefully from the most trustworthy of
Another swipe at Evergreen is that it is not a true open source project; that
its history as a top-down project initiated by GPLS means that there is no
real development community around Evergreen. If you've followed the Evergreen
mailing list, you wouldn't believe a claim like this, and you would
proclaim it a blatant lie. To disprove this claim, you just
need to browse through the open-ils-dev mailing list and look for the emails
with the subject keyword "PATCH" and you'll see that some of us have indeed
been contributing patches to the source code. Beyond that, you'll also see
that there are many volunteer contributors for install and configuration
support, documentation, creation of VMWare images. So how could someone make
such a claim about Evergreen and get away with it?
It's all about trusting "authorities", not checking sources, and integrity (or perhaps
a lack thereof). Here's an excerpted quote about Evergreen from the State Library
of Ohio ILS Options Discussion Meeting Minutes - April 24, 2007
The documentation for the process is very poor, which is typical because it is
the last thing developers are thinking about. ... The source code is open but
they don't really follow the "playground" rules for the open source production
Here's where you need to really know your sources and check your references.
Note that the claims about the nature of Evergreen as not being a true open
source project are credited to the introductory speaker, Stephen Hedges. Who
is Stephen Hedges? He was the director of Nelsonville Public Library (NPL)
when he worked with Joshua Ferraro to install Koha as the NPL integrated
library system. In addition, he is listed as the contact for Koha documentation
submissions. It seems, then, that he has a fairly significant personal stake in
the success of Koha, and if the meeting minutes accurately capture his
statements about Evergreen, it sounds like he was interested in dissuading
attendees from seriously considering Evergreen as an option. Subjectivity alert: as one of the volunteer contributors of code, documentation, install assistance, and a VMWare image of Evergreen from outside GPLS, this quote got me pretty hot under the collar; I've contributed to other open source projects, such as the Linux Documentation Project and PHP, and you always have to prove that you understand the project before being granted commit access.
Correction update: 2007/06/26Wow. In the following paragraph, I somehow made a very stupid mistake by incorrectly attributing Joshua Ferraro of LibLime with making statements about Evergreen at that meeting when he was not even present. All of the statements about Evergreen should have been attributed to Stephen Hedges. I apologize profusely to Josh and LibLime for this mistake.
Who was Stephen introducing as the guest speaker of honour on the subject of
open source ILS options in libraries? The speaker was Joshua Ferraro,
president of LibLime, the company best known for offering commercial support
for Koha. LibLime did announce that they would offer commercial
support for Evergreen, and have added sections about Evergreen to their Web
site, so it would on the surface seem to be a logical choice to invite a
LibLime employee as a one-speaker-fits-all host to cover both Koha and
Evergreen. However, LibLime has a rather unusual relationship with Evergreen.
It seems that LibLime has positioned Evergreen among their other offerings as
such a high-end product that only a handful of potential customers would qualify
for that market:
EvergreenFor consortia who need:* Scalability to hundreds of libraries, tens of millions of records
It sounds impressive, but way too high-end for the vast majority of
libraries. So of course people browsing the LibLime Web site will focus on the
Koha options instead. It seems like a deliberate bait-and-switch
move to attract libraries interested in Evergreen after the successful launch in
Georgia, but to get them to buy support for Koha instead. Consider: LibLime
has not contributed a single patch to the Evergreen development (open-ils-dev)
mailing list. LibLime has not contributed a single line of documentation to
the Evergreen wiki. LibLime does not include Evergreen among their demos.
LibLime hasn't made an Evergreen sale. So I think it's a fair question to ask
how committed LibLime really is to Evergreen - is LibLime's claim to support
Evergreen just a means to get people in the door, in hopes that they'll walk
out with a copy of Koha under their arms? I think so. You can come to your own conclusions.
In case you think that Ohio quote was just an unfortunate one-off, and that
I'm making a big deal about nothing, here's a more recent quote from the
Source Session Q&A of the "Everything You Ever Wanted to Know about Open
Source" conference held on June 6th, 2007 that caught my attention (and
which apparently no-one in attendance at the meeting was capable of providing
a rebuttal to):
Q. Contrast Koha & Evergreen?
A. Major difference: Koha was grassroots: started w/rural libraries, distributed organization, bottom-up decision making. Evergreen: PINES library system; top-down decision making. Koha: 800 libs worldwide, 8 years old; Evergreen: 1 year old, 1 consortium.
So there's the comment about the "top-down" nature of Evergreen again, and
this time Evergreen is being attacked for being immature and not very widely
used. (Note: on that very day, the British Columbia Ministry of Education
announced the BC PINES Website - so
another consortium is getting on board the Evergreen express.) If there really
are 800 libraries using Koha, I'm shocked at how many basic install, config,
and runtime problems are being reported on the Koha mailing lists with the
current 2.2.9 release... but I'm getting off-topic. The speaker was
[STRIKEOUT:once again] Joshua Ferraro, who:
... talked with us about open source integrated library systems,
specifically Koha and Evergreen, and about his company,
If your definition of "talking about" is "praise the product that pays your
bills and criticize the product that represents a major threat", then mission
accomplished. You can't blame the speaker for being in a perfect position
to pitch his product at the expense of a competing product, while being credited
with being an objective authority on both products. But I suspect the audience actually wanted a balanced presentation
about the two products.
So what's my point? Know your sources. If you invite someone to speak
on a broad topic, such as the State Library of Ohio meeting, where [t]he invitation was expanded to include any
library interested in the possibility of open source integrated library
systems (ILS), you might want to ensure
that any personal biases are very much out in the open for your audience
(both the in-person audience and the audience reading the meeting minutes at
home). If you're the speaker in such a situation, you should reveal any such biases.
If you're a company selling a
product or services related to a product, perhaps it's inevitable that the
profit motive is going to override ethics in such opportunities - but I can
dream. If you read the full minutes from the State Library of Ohio meeting,
you can see that in terms of an open source ILS option, Evergreen is given
only the most cursory coverage and the major focus is on selling Koha.
Getting a fair comparison
For a fair comparison of Koha and Evergreen, please consider either
hosting two separate presentations (you wouldn't consider asking SirsiDynix
to give a balanced presentation on all of the proprietary ILS options, would
you?), or try to find an independent speaker who can provide a more objective
analysis of the products at hand. Ask the speaker if they have any
financial ties to the products at hand. Heck, has anybody asked Marshall
Breeding and Andrew Pace if they've had any financial ties to ILS companies? I
assume the answer is no, but our community relies so much on their analysis of
the overall library systems landscape with so much financial implication for the
companies at question that it would be comforting to have a positive assertion
accompany any "state of the ILS landscape" articles in the future.
Ideally, you would find a member of the development community for each of
Evergreen and Koha. At the moment, I'm afraid that I can only qualify as a
member of the Evergreen community, but I plan to become more familiar with
Koha's codebase over the course of the summer - so maybe I can grow into that
position. Of course, then you would have to trust me. C'mon, you can trust me! **grin**
Make technology, not war
What would I like to avoid? I would really like to avoid negative energy being
invested in a Koha vs. Evergreen or LibLime vs. Equinox battle royale. That
doesn't interest me, but I'm sure it greatly interests the companies offering
proprietary products. Instead, I hope that this energy can continue to be
invested in making both Koha and Evergreen better by those with the technical
skills. Let's have a competition on product design and implementation, rather than
on marketing spin or dirty tricks. Everyone benefits from strong open source
library systems - even if you don't adopt an open source system, it raises the
bar for the proprietary systems to differentiate themselves.