Monica Brinkley provides an update on the EQUINOX project following on from an earlier article in Exploit Interactive. Progress on the development of performance indicators for electronic library services is reported. These performance indicators aim to enhance and complement the ISO 11620 library performance indicators. Management information software has been developed and tested by the EQUINOX project. The EQUINOX software tool is designed to help library managers implement quality management and performance measurement principles.
The EQUINOX project is a research project funded under the EC fourth framework Telematics for Libraries programme [1]. The project, now in its final phase, aimes to advance the field of quality management and performance measurement in European libraries. This has been achieved firstly by developing a set of performance indicators for electronic library services, which will enhance and update existing standards for performance measurement in libraries. And secondly by the project team developing a software tool for library managers, which aids and facilitates the implementation of performance measurement and quality management.
The EQUINOX project is a two-year project, due to be completed in November 2000. The project comprises both library and technical partners from a range of European countries. Background information about EQUINOX and its history are available from the project Web site [2] and in an earlier article in Exploit Interactive [3] that introduced the project. The current article reports the results and findings of EQUINOX to date.
Performance indicators provide library managers with a standard and manageable method of measuring the library’s performance as well as allowing benchmarking and comparison between libraries. Electronic library services have been increasing in importance in terms of the percentage of the library budget being spent on them and in terms of the ease of access that they provide to users. However, to date no standard method has been available for evaluating how this important section of library services is performing and whether value for money is being achieved. The EQUINOX performance indicators for electronic library services provide the library manager with a methodology for gathering valuable information about the quality of service being offered.
An initial set of performance indicators (PIs) was developed by the project team during the first phase of the EQUINOX project and published through the EQUINOX Web site [2], relevant mailing lists, the professional literature and at EQUINOX workshops. Feedback was received from a wide range of professionals from Europe and around the world. At the same time, the library partners in the EQUINOX project team tested these performance indicators in real library situations in terms of ease of data collection, applicability to the current situation in the library, relevance to library management, and so on.
Both the feedback received and the testing of the PIs showed that while this initial set was very useful, further revision of the PI list was required. Some PIs were felt to be too demanding of staff time in relation to the value of the information gleaned. Others were felt to be somewhat confusing and difficult to calculate. In many cases, the data required simply wasn’t available from many of the electronic library services currently in use in libraries. Following deliberation and discussion between project partners and with external interested parties and experts in the field, a revised set of performance indicators was developed by the project team. A discussion of the deliberations leading to this revision is available on the project Web site [4].
The revised set of performance indicators, together with a definition of terms used is listed below. The full definition and collection methodology for each PI is available on the project Web site [5]. The final phase of the project will further refine these performance indicators to reflect feedback received; comments are therefore welcome even at this late stage of the project, directly to the author or through the project Web site.
It must be noted that these PIs are designed to enhance and complement rather than to replace the ISO 11620 Library Performance Indicators [6]. The ISO 11620 indicator ‘User Satisfaction’ in particular should be measured in conjunction with these PIs. For example, user satisfaction with each electronic library service and electronic library services generally would provide enhanced qualitative management information, in particular when considered in conjunction with PIs such as numbers 2 and 5 below. By considering pairs of PIs together in this way the manager can elicit even more valuable information. For example, the cost per session of a particular service considered in conjunction with the number of documents downloaded per session can give a better indication of value for money than has been available heretofore.
The term ‘library’ is used throughout but can be taken to refer to an information service irrespective of its title.
Standard definitions have been used where possible. Whether the definition is taken from an ISO document or has been defined by the EQUINOX project is indicated in bold after the definition.
Cost: Acquisition, subscription and license costs for electronic library services made available by the library. Network and hardware costs should not be included. EQUINOX
Document: Recorded information or material object which can be treated as a unit in a documentation process. ISO/FDIS 5127 [NOTE: Documents can differ in their physical form and characteristics.]
Downloading transaction: Any procedure that aims at reproducing electronic data onto a local storage medium or printing facility. ISO/DIS 2789
Electronic library resources: Every document in electronic form which needs special equipment to be used. Electronic resources include digital documents, electronic serials, databases, patents in electronic form and networked audiovisual documents. ISO/DIS 2789
Electronic library services: A service which is either supplied from local servers or accessible via networks. Electronic library services comprise the OPAC, the library Web site, electronic resources, electronic document delivery and internet access offered via the library. ISO/DIS 2789
Entry: Bits of information, especially from reference databases or directories e.g. records, abstracts etc. ISO 5127-1
Information request: User enquiry that involves the knowledge or use of, or recommendations, interpretation or instruction in the use of, one or more information sources (such as printed and non-printed materials, machine-readable databases, the library’s own and the institutions’ catalogues) by library staff. The request can be delivered personally or by means of telephone, regular mail, fax or electronic media. Do not include directional or administrative enquiries e.g. locating staff or facilities, opening times or handling of equipment. Do not include enquiries about locating items in stock that have already been identified bibliographically. ISO/DIS 2789
Library computer workstation: Public access networked and stand alone computers, provided by the library, though not necessarily in the library, offering access to electronic library services. EQUINOX
Session: An established connection to an electronic service, usually by a log-in. [NOTE: connecting to a Web site is regarded as a session if its referring link is external to the Web site. Sessions to a general entrance or gateway page should be excluded.] ISO/DIS 2789
Population to be served: Number of individuals for whom the library is set up to provide its services and materials. For public libraries this will normally be the population of the legal service area: for academic libraries this will normally be the total of academic and professional staff plus students. ISO 11620
Rejected session: Unsuccessful session to an electronic library service which is not available because of requests exceeding the simultaneous user limit. [Note: Rejection of wrong passwords is excluded.] ISO/DIS 2789
Target population: Groups of actual and potential users appropriate to an individual library as the object of a specific service or as the primary users of specific materials. The target population may be the population to be served. ISO 11620
In conjunction with the development of performance indicators for electronic library services, the EQUINOX project team have developed a software tool to aid the library manager to implement performance measurement and quality management practices in their library.
The EQUINOX system works on the basis of a quality framework, which is devised by each library to reflect their local situation. This framework consists of one or more missions, out of which arise aims. Objectives are defined relating to each of these aims, and the objectives are measured in terms of specific performance indicators (ISO 11620, EQUINOX, or locally defined indicators).
 Figure 1: Quality Framework in the EQUINOX System |
For example, Figure 1 above shows one of the library’s missions as being: ‘To support the teaching and learning activities of academic staff and students’. One of the aims arising from this mission is: ‘To equip members of the institution with the skills needed for the Information Society’. One of the specific objectives devolving from this aim is: ‘To maximise the use made of the electronic library’. The EQUINOX system can measure and record whether this objective is being met, through the performance indicators which have been defined as constituting this objective, namely: ‘Percentage of the population reached by electronic library services’ and ‘Number of sessions on each electronic library service per member of the target population’. Each of these is calculated from the datasets shown in Figure 1 and according to a predefined formula. Target values, minimum and maximum thresholds can also be defined by the library. The system compares the value of a PI or objective to these thresholds and an alerting facility will alert the relevant staff if a threshold has been exceeded.
This quality framework or hierarchy and its constituent entities can be stored, viewed and edited with ease through the EQUINOX software. Colour coding distinguishes clearly between different levels in the hierarchy. Entities can be viewed in their own right or as part of the hierarchical structure. Links to qualitative or related documentation can be provided from entities anywhere in the hierarchy and will be automatically launched using the appropriate software when selected. For example, links might be provided to service level agreements or to an analysis of a user survey.
The EQUINOX software consists of two modules: an administrative module and a review module. The administrative module is used to input the hierarchical structure and to set up the system according to the library’s needs. This module is also used to input actual data relating to library activities and performance, which is then used by the system to calculate performance indicators and objectives. The review module is aimed at library managers. This module allows the library’s quality framework to be viewed and printed. The performance of the library can be assessed by viewing or printing graphs depicting the values of a specific performance indicator or objective over time, as shown below. These can also be copied for inclusion in reports.
 Figure 2: Graph of PI: Percentage of the population reached by electronic library services |
 Figure 3: Graph of Objective: To maximise the use made of the electronic library |
The EQUINOX system has been tested by the partner libraries in live library environments. These tests resulted in minor changes to the system followed by a second (or beta) test phase. The beta testing resulted in further enhancements being made to the system. Overall the libraries found the system easy to use and extremely useful for managing quality and performance measurement data. Forty European libraries and two libraries in the US are currently testing the final version of the EQUINOX software. This large scale demonstration trial is due to be completed during September 2000, and feedback received from these trials will feed into plans for the future exploitation of the EQUINOX system.
Overall the EQUINOX project has been a very successful project. Two valuable additions to the area of performance measurement and quality management in libraries have resulted from this project. Performance indicators have been developed and tested for electronic library services to enhance and complement the existing standard indicators for traditional library services. These have been disseminated to and discussed with an international audience, ensuring their relevance to the profession and their position at the leading edge of developments in this field. A prototype software tool to aid the library manager attempting to implement quality principles in his or her library has been developed and tested and has proven to be not only possible but extremely useful.
The remaining phase of the EQUINOX project will include consideration of future exploitation of these two products of the project. The performance indicators have already been discussed with the relevant ISO Committee and may be considered for future revisions of ISO 11620. The project has identified a clear need and market for a software product such as EQUINOX and commercial exploitation of this product is being investigated.
Monica Brinkley
Research Officer
Dublin City University Library
Dublin City University
Collins Ave, Dublin 9
Ireland
Email: mbrinkley@esatclear.ie
URL: <http://equinox.dcu.ie/>
Monica Brinkley is the EQUINOX Research Officer for Dublin City University Library. She has previously been involved in the research project BIBDEL: Libraries without Walls, which investigated the delivery of library services to remote users. Monica has worked both as an academic librarian and a lecturer in Library and Information Studies.
For citation purposes:
Monica Brinkley, "Performance Measurement and Quality Management for the Hybrid Library: An update on the EQUINOX project", Exploit Interactive, issue
7, 2nd October 2000
URL: <http://www.exploit-lib.org/issue7/equinox/>
[HTML Validation] - [Accessibility check]
|
Issue Home | Editorial | Features | Regular Columns | News and Events | Et cetera | ||
|
| ||
| Go to Top |
A UKOLN Service. Contact Us. Copyright © 1999 |
Last Updated: 2nd October 2000 |