Perceptions 2008: An International Survey of Library Automation

by Marshall Breeding. January 18, 2009

Introduction

This report describes the results of a survey that I conducted to gather data regarding the perceptions of libraries toward their automation systems, the organizations that provide support, and the quality of support they receive. It also aims to gauge interest in open source library automation systems.

I conducted a similar survey in 2007.

This year, I received 1,450 responses from libraries in 51 different countries. The countries most strongly represented include the United States (1,150 responses), United Kingdom (49), Canada (99), Australia (44). As with the general demographics of the lib-web-cats database, the respondents of the library primarily come from libraries in English-speaking countries. Survey results were gathered between October 31, 2008 and January 16, 2009.

The survey attracted more responses from libraries using Millennium (293), Unicorn/Symphony (233), and Horizon (206). There were fewer than 100 responses for each of the other ILS products represented in the survey. Systems with less than 20 responses did not appear in the main statistical tables. These responses can be seen through the individual ILS Product Reports available.

Note: In this report I use the name Unicorn to represent the ILS product from SirisDynix currently marketed under the name Symphony that was previously called Unicorn.

This article is an original publication of Library Technology Guides and is not slated to appear in any print publication. Please direct any comments or enquiries to the author.

This survey and its analysis reflect my ongoing interest in following trends in the library automation industry. It is designed to complement, and not replace, the annual Automation Systems Marketplace feature that I have written the last seven years for Library Journal. The survey underlying the Library Journal article relies on information provided by the companies that offer library automation products and services. The survey that serves as the basis for this article collects data from the libraries themselves.

Survey Results

An interactive version of the statistical reports is available here, which includes the ability to view the responses for each of the ILS products, along with the redacted comments.

---

Most Positive Perceptions

Polaris emerged this year as the ILS product with the highest positive ratings in the categories of product and company satisfaction. Libraries using VERSO from Auto-Graphics gave the highest rankings for customer support and loyalty to the company for future business. Library.Solution from The Library Corporation received highly positive marks from its customer libraries in all categories. Libraries using Polaris, VERSO, and Library.Solution showed the least interest in open source ILS products. These three companies received extremely high satisfaction ratings from their libraries, with average scores separated by very thin margins.

Negative Perceptions

The survey results reveal high levels of dissatisfaction by libraries running legacy ILS products. Athena and Winnebago Spectrum, both systems acquired by Follett Software Company that will not receive ongoing development, received the lowest ILS satisfaction scores and indicated the least likelihood that they would purchase an ILS in the future from this company. Libraries using Dynix gave low marks regarding their satisfaction with the product (5.14) and for SirsiDynix as a company (4.81), but rated support more moderately (5.76). Horizon libraries gave SirsiDynix very low marks as a company (4.32) but registered moderate satisfaction for the product itself (5.68).

The number of negative comments provided on the survey forms overwhelmingly exceeded positive ones.

ILS Satisfaction

Polaris ranked as the product that received the highest score in response to the question probing satisfaction with the library’s Integrated Library System with a median rating of 7.73. Fifty-one libraries using Polaris responded to this question. Last year a total of 59 responders rated Polaris 7.78, reflecting remarkable consistency across the two years. VERSO earned second highest marks in this category (7.26), with Library.Solution from The Library Corporation only a fraction lower (7.20). Millennium from Innovative Interfaces, Inc. also attracted highly positive ratings (7.09).

Company Satisfaction

Polaris Library Systems also won the highest score for company satisfaction (7.76) with Auto-Graphics (7.68) and The Library Corporation (7.33) only slightly less favored. Libraries using Millennium gave Innovative Interfaces solidly positive ratings (6.44), though a notch below the top three companies. Libraries using legacy products not surprisingly noted their vendors as least satisfactory, including those using Athena (3.92), Horizon (4.32), Winnebago Spectrum (4.52), and Dynix (4.81). The middle tier of company satisfaction included those using Koha supported by LibLime (5.84), Virtua from VTLS (5.79), Voyager (5.59) and ALEPH 500 (5.20) from Ex Libris, and Unicorn from SirsiDynix (5.05).

Satisfaction with Customer Support

Libraries using VERSO rated Auto-Graphics as the company providing the most satisfactory support (7.81). Polaris (7.41) and The Library Corporation (7.07) also earned highly positive ratings for customer support, just below that of Auto-Graphics. Innovative received strong marks in this category (6.46), though again just a notch below the top tier. Users of Athena (3.63) and Winnebago Spectrum (4.57) gave Follett low ratings for support.

Company Loyalty

In response to the question probing the likelihood that the library would purchase future ILS products from their current vendor, Auto-Graphics received the highest marks for customer loyalty (7.64) only slightly edging above The Library Corporation (7.50) and Polaris Library Systems (7.33). Libraries using Millennium gave mixed results, but overall indicated strong loyalty to Innovative Interfaces, Inc. (6.54). Libraries using Athena (4.32), Winnebago Spectrum (3.95), and Horizon (4.37) seem on average not inclined to purchase their next system from their incumbent vendors.

Open Source Perceptions

It’s not surprising that the libraries already using an open source ILS registered the strongest interest in future consideration of an open source ILS, with Koha as supported by LibLime toping the list (8.05). Other than these open source true believers, libraries running proprietary systems submitted responses reflecting much lower interest, with even those most dissatisfied with their current product such as Winnebago Spectrum (4.95) indicated relatively weak interest. We also observe that libraries most satisfied with their current situation, including Polaris (2.29), VERSO (2.63), Library.Solution (3.00) showed little interest in open source alternatives.

One of the major movements in the library automation industry in the last few years involves the entrance of open source ILS products as a mainstream option. That libraries using these products now appear in this survey reflects that this approach has made inroads among the long-established proprietary systems. The three open source ILS products represented in the survey results include OPALS, Evergreen, and Koha, though only Koha received sufficient responses for inclusion in the primary tables.

Libraries using Koha with support from LibLime indicated only moderate satisfaction with this approach. These libraries placed Koha in the middle of rankings for ILS satisfaction (5.92), rated LibLime only moderately as a company (5.84) and for its quality of support (5.52). Those contracting with LibLime with support for Koha also indicated moderate loyalty (5.24) in dealing with the company for their future ILS.

In all categories other than the interest in open source ILS, this set of libraries gave less positive satisfaction ratings than those using Polaris, VERSO, Library.Solution, and Millennium.

---

Comments on Selected Companies and products

Polaris Library Systems received extremely positive ratings from its customer libraries. The Polaris library system finds use almost exclusively in public libraries. Last year its customers gave the company the highest satisfaction ratings in all categories. The company was edged out in the ILS Support and company loyalty questions, though by only very thin margins. The comments given by respondents using Polaris were overwhelmingly positive. A couple of comments indicated a slight diminishment in the quality of support over the last year—corroborating the slightly lower ILS support perceptions score this year (2007 = 8.11, 2008 = 7.73). 88.7 percent of libraries using Polaris reported that the system was installed on schedule.

Auto-Graphics, which barely showed up in last year’s survey, received extremely impressive customer ratings this year. 81 libraries using VERSO responded to the survey, and most seemed to be delighted with the product, the company, and the service provided. Auto-Graphics provides this software almost exclusively through the Software-as-a-Service model. Almost all of the libraries using VERSO that responded were small public libraries. These libraries generally come to VERSO as their first automation system or are migrating from PC-based products such as Athena, Winnebago Spectrum, or Circulation Plus. 96 percent of the libraries using VERSO indicated that the system was installed on schedule.

The Library Corporation received strongly positively ratings from its customers using Library.Solution. It earned the second highest score in response to the question measuring loyalty to the company when purchasing a new ILS and third for the questions dealing with satisfaction to the ILS, with the company, and with customer support. The responses from libraries using Carl and Carl.X were too few to be included in the main survey result tables and were considerably less positive than those using Library.Solution. The libraries using Carl, the company’s legacy system gave a strongly positive response (6.55) for company loyalty, which should be good news to The Library Corporation in retaining libraries using this legacy product. 96.7 percent of respondents reported the system was installed on time and only one library indicated that it was considering migrating to a new ILS.

Innovative Interfaces received generally positive scores for its Millennium ILS, placing fourth from the top in the four satisfaction categories. A total of 293 libraries responded to the survey—more than any other product. Given the high number of responses coupled with the larger size and complexity of the libraries responding, the results provide a positive impression of the company. Many libraries using Millennium provided comments expressing a very wide range of opinions. Many were positive, noting the strengths and the stability of the product. Some complained about the cost of the software or its lack of openness. 92.7 percent of Millennium libraries reported that the system was installed on time. 8.31 indicated some interest in migrating to a new system.

Ex Libris supports two of the systems represented in the survey results: Aleph 500 and Voyager both implemented primarily in academic libraries. Ex Libris created Aleph 500 while Voyager was acquired from Elsevier. The company continues to develop and support both products. Aleph 500 tends to be used in large and more complex implementations. In all four satisfaction categories, these libraries fell in the middle tier of response levels. It’s interesting that in all of the satisfaction categories that libraries using Voyager responded more favorably than those using Aleph 500. Many libraries commented that the support for Voyager has improved since its acquisition from Elsevier. Voyager libraries expressed stronger interest than those involved with Aleph 500.

VTLS was ranked in the middle tier of satisfaction from the libraries that responded to the survey that use its Virtua ILS. Thirty percent of these libraries using Virtua indicate some interest in migrating to a new ILS.

SirsiDynix controls three of the ILS products represented in the survey results: Unicorn, Horizon, and Dynix. Unicorn, known more recently as Symphony, is the longstanding product developed by Sirsi Corporation and stands as the strategic flagship ILS of SirsiDynix. Horizon, gained through the acquisition of Dynix, will be supported in the short term, but will be phased out in the longer term. Dynix Classic, an older ILS product, has been considered a legacy product for several years and has a rapidly diminishing number of installations. Given the relative positioning of the products by SirsiDynix, some of the perceptions of the libraries using these products provide some interesting contrasts. Interestingly, libraries using Dynix (5.76) or Horizon (5.35) gave SirsiDynix higher marks for support than did those using Unicorn (4.91). It seems remarkable that libraries running the company’s strategic ILS seem significantly less satisfied with support than those involved with the products being phased out. When it comes to loyalty to the company for purchasing its next ILS, libraries using Dynix gave higher ratings (5.20) than those running Unicorn (4.95) or Horizon (4.37).

Given the legacy status of Dynix and Horizon, it’s not surprising that over 60 percent of libraries running these systems indicate an interest in migrating to a new system. Rather, I wonder why the other 40 percent aren’t more actively pursuing their options, especially those running Dynix Classic. Of more concern to SirsiDynix should be the 23 percent of libraries running Unicorn that indicate an interest in migrating to a new system. Libraries currently using Horizon expressed strongest interest on open source ILS other than those already using one.

The comments given by respondents from libraries running Unicorn, Horizon, and Dynix express a number of major concerns about SirsiDynix as a company, with mixed opinions regarding the quality of the products. Comments from libraries running Horizon focused on concern with the company more than with dissatisfaction with their current product. Libraries running Unicorn gave mostly negative comments, with only a few exceptions.

Follett Software Company owns several ILS products represented in the survey including Athena, Winnebago Spectrum, Circulation Plus, InfoCentre, and Destiny, though the latter two did not receive sufficient responses to be included in the main response tables. Destiny stands as the company’s one flagship ILS product while all of the others have been designated as legacy products. The company received strongly negative satisfaction ratings from libraries using Winnebago Spectrum, Athena, and Circulation Plus. We note that Follett Software Company specializes in school libraries and the vast majority of the responses in this survey were from public libraries using its products. Only 34 responses from school libraries compared to 730 from publics. Libraries using Follett’s Destiny product registered extremely positive ratings. Although the number of responses was below our threshold to be presented on the main tables, libraries using Destiny rated Follett with a superlative average score of 8.25.

---

Details about The Survey

The survey instrument included five numerical ratings, three yes/no responses, and two short response fields, and a text field for general comments. The numeric rating fields allow responses from 0 through 9. Each scale was labeled to indicate the meaning of the numeric selection.

Four of the numeric questions probe at the level of satisfaction with and loyalty to the company or organization that provides its current automation system:

• How satisfied is the library with your current Integrated Library System?
• How satisfied is the library overall with the company from which you purchased your current ILS?
• How satisfied is this library with this company’s customer support services?
• How likely is it that this library will purchase its next ILS from this company?

A yes/no question asks whether the library is considering migrating to a new ILS and a fill-in text field provides the opportunity to provide specific systems under consideration. Another yes/no question asks whether the automation system currently in use was installed on schedule.

Given the recent interest in new search interfaces, a yes/no question asks “Is the library currently considering a search interface for its collection that is separate from the ILS?” and a fill-in form to indicate products under consideration.

The survey includes two questions that aim to gauge interest in open source ILS, a numerical rating that asks “How likely is it that this library would consider implementing and open source ILS?” and a fill-in text field for indicating products under consideration.

The survey concludes with a text box inviting comments.

View the survey. (This version of the survey does not accept or record response data.)

In order to correlate the responses with particular automation systems and companies, the survey links to entries in the libraries.org directory of libraries. Each entry in libraries.org indicates the automation system currently in use as well as data on the type of library, location, collection size, and other factors that might be of potential interest. In order to fill out the survey, the responder had first to find their library in lib-web-cats and then press a button that launched the response form. Some potential respondents indicated that found this process complex.

The link between the lib-web-cats entry and the survey automatically populated fields for the library name and current automation system and provided access to other data elements about the library as needed. The report on survey response demographics, for example, relies on data from lib-web-cats.

A number of methods were used to solicit responses to the survey. E-mail messages were sent to library-oriented mailing lists such as WEB4LIB, PUBLIB, and NGC4LIB. Invitational messages were also sent to many lists for specific automation systems and companies. Where contact information was available in lib-web-cats, and automated script produced e-mail messages with a direct link to the survey response form for that library.

The survey attempted to limit responses to one per library. This restriction was imposed to attempt to sway the respondents to reflect the broad perceptions of their institution rather than their personal opinions.

The survey instrument was created using the same infrastructure as the Library Technology Guides web site—a custom interface written in perl using MySQL to store the data, with ODBC as the connection layer. Access to the raw responses is controlled through a user name and password available only to the author. Scripts were written to provide public access to the survey in a way that does not expose individual responses.

In order to provide access to the comments without violating the stated agreement not to attribute individual responses to any given institution or individual, an addition field was created for “edited comments.” This field was manually populated with text selected from the “comments” text provided by the respondent. Any information that might identify the individual or library was edited out, with an ellipse indicating the removed text. Comments that only explained a response or described the circumstances of the library were not transferred to the Edited Comments field.

Statistics

To analyze the results, a few scripts were written to summarize, analyze, and present the responses.

In order to avoid making generalizations based on inadequate sample sizes, the processing scripts included a threshold variable that would only present results when the number of responses exceeded the specified value. The threshold was set to a value of 20.

For each of the survey questions that involve a numeric rating, a set of subroutines was created to calculate and display simple statistics.

• Responses indicates the number of survey responses that made a selection for this question.
• A Response Distribution array lists the number of responses for each possible value from 0-9.
• The Mode indicates the numeric response that receive the most selections.
• The Mean is the average response, calculated by adding together all the responses and dividing by the Responses value, rounding to two significant decimal places.
• The Median is the middle response, calculated by placing each of the responses in a sorted array and selecting the middle value.
• The Standard Deviation was calculated by subtracting each response value from the mean, squaring the difference, summing the squares and dividing by the number of responses to determine the variance. The standard deviation is the square root of the variance.

The “report-by-category.pl” script processes each of the numerical ratings, displaying each of the statistical components listed above for each product that received responses above the threshold value. This report provides a convenient way to compare the performance of each ILS product for the selected question. The report sorts the statistics for each product in descending order of the mean. The report categories available correspond to the survey questions with numerical scale responses.

The product-report.pl” script provides the results for each of the ILS products mentioned in the responses. This report also provides the statistical components for each of the numeric question. It also provides the percentage of yes responses to the two yes/no questions:

• Is this library currently considering migrating to a new ILS?
• Is this library currently considering acquiring a search interface for its collection that is separate from the ILS? The report then lists the “EditedComments” text.

[The text of this section mostly replicates what appeared in the 2007 version of this article. For for both editions of the survey I followed the same methodology for collection and and statistical analysis.]

---

Caveat

As I noted with last year’s survey, one should not read too much into the survey results. Responders to the survey provide their subjective impressions to fairly general questions. Although the survey instructions encourage responders to consider the broader institutional perceptions, it’s usually the case that multiple opinions prevail within any given library. While I believe that this survey does provide useful information about the experiences of libraries with their current integrated library systems and the companies that provide support, it should not be used as a definitive assessment tool.

Maintained by Marshall Breeding Copyright 1999-2023