Note: over on Techsource, I wrote about open source OPACs this month — worth reading if you’re thinking about OPACs and such. Also see Marshall Breeding’s Library Technology Reports on Next-Generation Library Catalogs.
A week from Friday, I’m giving a talk in Illinois called “The OPAC sucks” at at the Symposium on the Future of Integrated Library Systems (yes, I’m getting a little tired of the “suck” meme, but it’s my fault for starting it).
I’d like your input.
Last year I discussed some of problems with OPACs in a three-part series for Techsource, in which I described problems with ranking, spell-check, display, and other issues. (Here’s a follow-up post that links back to Techsource while correcting one point. In fact, if I could rewrite the original posts I would say “ranking” rather than “relevance ranking.”)
Has your OPAC unsucked, even a little, or are you planning to unsuck it? Are you more aware of your OPAC’s limitations — perhaps through usability studies, user focus groups, or search log analysis? Are you thinking about substituting your OPAC with something else? Are you putting your focus (and your resources and money) elsewhere? Are you thinking about open source solutions?
What about some of the bigger issues, such as data formats, invisibility on the open Web, etc. — do you see solutions for these problems? What about the idea of getting out of our “institutional silos” and becoming part of one massive database — a la OCLC’s WorldCat Local? What are the threats and opportunities?
Posted on this day, other years:
- Excerpt, Essay 13, David, just as he was - 2006
- FRL: Post Away! - 2005
- The UnMuzzled Media - 2005
I’ve been doing a LOT of search measurement recently (database of 18M search clicks), so my first thought is, “how do you measure suckitude?”
Libraries are both in a stronger and weaker position for measurement than WWW search (like Google). You have a very strong success signal — someone checked out the book — but a weak connection between the search and the checkout.
I’d look at two measures: percentage success on searches, and percentage of checkouts that are initiated in search.
One quick test would be to watch the search terminals for an hour and do a count on number of patrons using them and number of patrons checking out books. Session length would be another good thing to observe.
Hmm, while you are at it, read the first part of Tom Laundauer’s “The Trouble with Computers”. He talks about the inverse relationship between IT spending and productivity, and why that was different in telecom. The short version was that they only installed technology that measured as more productive.
I’m in a project group that’s just starting to look at our catalogue to see what we can do in the short term (ie by the end of the year) by way of tweaking the (HIP but not very hip) system we already have. Essentially quick, simple, cheap and useful fixes. On the table is everything from implementing simple web usability tweaks to adding features like spelling correction, user ratings/comments/tags, RSS feeds etc – but it’ll all depend on what the system will allow in the time we have available.
I’m working on setting up usability testing at the moment – it’s going to be a bit rushed due to time constraints but any usability testing is better than none.
Everything we can’t do this year will go on the wish list for a project next year to work out if we want to migrate to another system entirely and if so what. (A number of us are agitating for Aquabrowser…) This will be followed by finding out whether or not we have the budget for it. I get the impression that current open source software isn’t at a stage where it’s suitable for us – of course this might change.
Walt, I’ll look at that book. In terms of behavior, is it worth asking what kind of searching people are doing? It’s hazarded that OPACs are best at known-item searching.
Deborah, Steven Krug (“Don’t Make Me Think”) concurs with “any usability testing is better than none.” Just don’t limit yourself to librarian subjects 😉
Spell-check is a biggy. When we implemented that in a former Place Of Work, it had a huge impact on success. I wouldn’t call it a “small” thing to do, but in terms of outcomes, the payoff is huge.
When you say talk about open source software not being “suitable,” have you compared it bolt by nut by washer with vendor software?
I almost added “ask about their user testing”. If it hasn’t been tested for something, it isn’t good for that.
I always assumed that OPACs were stuck on “the perfect 30 item Boolean search”.
Searcher goals (known-item, informational) are useful, and not that hard to figure out. List the top 200 (or 500) queries for the week and go down the list marking them as title, author, or topic. It doesn’t take that long. Do it in a spreadsheet and use IFSUM to count up the occurances of each type to get percentages. Then use that to decide what to fix first.
If informational searches are 3% of your queries, then tags won’t make a big difference. At best, they can make a 3% difference. In reality, some of those searches already work OK and tags will only help some of the rest.
You will probably find a few dominant informational searches. Out here, it would be “California Missions” at any library that serves 4th graders. Pickle that answer in an editorial category. Mine queries for those special displays and handouts.
Spelling errors are about 10%, probably the biggest single slice of query behavior that you can address with a targeted feature. The errors are often concentrated on a few titles. “Ratatouille” is nearly unspellable, “Shooter” is easy. If you can help half of the misspelled, you’ve gained five percentage points.
I don’t have any control over our OPAC, so there’s not much I can actually do, but I have dreams.
I think we have to start by recognizing that the goals for one kind of OPAC are not the goals of another. Public library patrons looking for children’s books and academic library patrons looking for resources for a paper are making radically different uses of the same system, and at present it rarely serves either of them very well.
I have lately been on a rant about the difficulties of searching for children’s books in catalogs. In many libraries, children’s books are divided into several call number sections (picture books, easy readers, beginning chapter books, etc.), but these divisions are rarely searchable in the catalog. Even in my tiny library, searching for a children’s book about princesses gives you a huge list from which you must cull out the picture books or the nonfiction books or whatever it is that you’re actually looking for. (And then, of course, there are the subject headings for children’s books–“Juvenile fiction” means fiction, but to get nonfiction, you have to use “Juvenile literature.” It’s insane.
There are improvements, of course–having book covers available for children’s books is a great boon. But there are so many more things that could be done. I’d love to see a mashup of the OPAC and A to Zoo, for instance. Teachers often want a bunch of picture books on a particular subject, and we all spend a lot of time compiling such lists, since regular subject headings are often not very helpful for, say, a group of books about honesty or starting kindergarten. Since that kind of work has been done, it seems a shame that it’s not more usable.
So those are my thoughts. I’d like to see us come up with wonderful new catalogs, but in the meantime, I’d like to see us using all the data we already have and making it work a little harder.
[…] Karen Schneider has written a pretty good overview of open source software and the reasons why it’s gaining in popularity among librarians. She follows up her TechSource column with another on her own blog. […]
[…] Search for an item in libraries near you:Enter title, subject or author WorldCat.org >> ← Your OPAC and the Suck Factor […]
Actually I’ve just recently finished reading Krug’s “Don’t Make Me Think” so that probably influenced my thoughts and/or phrasing there. Brilliant book.
I don’t know exactly how in depth anyone here has looked at open source options – I’m not in IT myself and though I asked about it briefly the other day it wasn’t really in the scope of the meeting so it was brief. My impression was that it was to some extent a toss-up between having a system provided by a company which may then leave us hanging (staring out at the horizon not looking at any company in particular) and a system that needs a lot of extra customisation/features. From the limited reading I’ve done of Koha, it seems to be mostly used by smaller public libraries than by medium-sized academic libraries – but I haven’t explored the subject far enough to see whether that’s inherent in the system’s capabilities or inherent in the fact that change is harder for larger organisations (or both or neither).
Krug is a nice guy, too.
My feeling about companies is they can all leave us hanging… you don’t want to implement something that clearly can’t do what you need it to do, but sometimes open source software can be as good or better than its competing products. Open source ILS software has two strong maintenance companies, and more could emerge. Food for thought…
Yours may suck. Mine blows.
I once shared an office with a retired USN sub guy who had a bumper sticker on the wall above his desk: “Pneumatics suck and blow.”
Been online too long…stepping s-l-o-w-l-y away from the computer now…
I am not happy with our OPAC but we are limited in what we can buy and do because we are a two person special library. IT doesn’t allow us to have our own server so we rely up on an ASP hosted service. I don’t have time or programming knowledge to tweak a system. I would love for there to be more options but I would also love for those things to be availble or possible for small libraries. It seems that so many of the Open source ILS solutions seem to be more directed at larger libraries who have people with programming knowlege.
Webdoyenne, when I was a jet engine mechanic, I was taught that the basic principles of jet engines were “suck, squeezes, blows, and goes.” Naughty, but on point!
Michelle, I’ve worked in a small special library and have known the frustration of limited services. (At least yours is an ASP. Ours was hosted by the company IT, who one week accidentally deleted our serials database.) Some small libraries are using open source. It would be interesting to hear from them.
My law firm library uses a hosted solution. It works surprisingly well. Number 1 we almost never have to wait for our busy IS (Information Services) Dept. to perform any magic.
Quarterly updates are regular and mostly improvements. Help is a phone call away.
We have some control over the look and feel of the OPAC.
The vendor is somewhat responsive. So far we have been able to get them to partner with an RFID vendor to integrate with their circulation system.
We are now working on another vendor for RSS that seems to be coming along in the development stages.
The Development Director at the ILS vendor speaks with me as often as I call.
I would not have selected it but it was purchased before I was hired and I was charged with implementation.
This OPAC almost does not suck – compared to the other options. Most people do a quick keyword search and if they don’t find what they want, call the library anyway. Very different than public or academic library patrons I assume.
A big negative is they cannot at this time provide us with user statistics.
I think Walter makes the classic non-librarian mistake, that checking out a book is the ultimate goal. It’s certainly an easy measurement, but there are so many other uses for OPACs (gauging topics, reading in the library, asking the reference librarian, etc.) that it’s important to include them in any discussion.
That said, hooray for search logs! You can learn so many things you don’t expect.
Karen, I would love to hear from other small libraries who do open source. I think there should be a small libraries movement where those one and two person libraries can share those one or two quick and easy things that we do (open source, TOC’s RSS’d, chat ref, whatever) that work on a s-m-a-l-l library scale. I know there are those of out there doing some things on our level, I just think our little voices are sometimes accidently drowned out by voices in bigger places.
Avi, concur! I just had a walkthrough of the latest Polaris and was taken by all the great features.
Michelle, that would make a terrific column… The Very Very Small Library Does Technology. I’ve managed two small libraries, and those who have met me in person know I’m also a small librarian 😉
Of course there are lots of uses of an OPAC, but I bet that finding specific items is the #1 use and that checking those items out is the #1 outcome in most libraries. If the OPAC sucks for the #1 use case, that’s bad. If we don’t know what the #1 use case is, we can’t fix it.
We just finished a round of usability testing on our OPAC (we use Polaris PowerPAC). While I don’t love everything about our online catalog one nice new feature is the ability to set up filters for searching. Our ILS lets us create Boolean searches and then package each search query in user-friendly drop-down. We divided up this drop-down menu to allow the customer to search everything in a material type (e.g. all Audio books), a subset of that material type (only Audio books on CD), and going even further, only a subset owned by a specific building (Audio books on CD only at the Main Library).
It’s not much, but if I can lead customers to this one little element that now sits on our catalog search screen, I’ve got to believe I’m going to make someone’s day.
http://catalog.cmpl.org/POLARIS/search/
The #1 problem I see with the search interface in our OPAC is this:
It’s a search interface.
Right there, you have hamstrung the casual browser.
The #2 problem is this:
It’s an OPAC (i.e., it contains metadata about books).
Right there, you have hamstrung the person who’s looking for something else, like magazine articles.
The #3 problem:
Neither relevancy ranking nor spell checking.
This stops the patron, not at step one, but right afterward. Either there’s no search result, or the top results have nothing to do with what the patron wanted.
We do have a few bells and whistles on the horizon for our OPAC result displays (possibly including pulling in LibraryThing data, adding book jackets, adding format icons). All these are perfectly nice solutions to some problem that someone is no doubt having. However, they do nothing at all about the above listed items that in many cases stop patrons cold before they ever see a search result.
The kinds of questions many patrons ask librarians here in person imply they’d be well served by virtual shelf-browsing (particularly with filters available — all movies on DVD, all children’s books, all photographs). This is partially implemented in our existing OPAC: you can in fact browse all DVDs, but the feature’s not very obvious; and to do a good implementation of a photo browse, we’d probably have to go beyond the OPAC and add on a digital library package.
I review our web site search logs (the main public site, not the OPAC web front-end). I find roughly a third of searches are author or title (known item); roughly a third are topics best searched in a magazine database or general web search engine; and the rest are actually searches on library information.
Similarly, I’d guess many of the OPAC searches are for information types or topics that just are not part of the OPAC data. (I don’t have easy access to those search logs.)
Federated search helps with that pesky problem of having to know where something is before you can start to look for it, but I don’t think that is on the near-term horizon either.
Relevance ranking and spell-checking, possibly coming sooner than federated search.
Well, I don’t make these decisions.
Good luck on your presentation tomorrow, Karen! My only comments, that I can make publicly, are as follows. If your current system is something you are unhappy with, take your time in picking something else. Do not rush into something else just for the sake of “change.” Yes, your current system stinks. But that doesn’t mean that the next one will be better. Remember Ranganathan’s Laws, apply them to decisions about your catalog system, and listen to your staff when making decisions that affect it: the single biggest point of contact we have with our users.
OCLC online library sucks. I’LL just keep stealing media. ciao