Thinking Out of the Meta-Box

(Update: Part 1 of at least 2 posts.) This is a long, nerdy post about search engines, so if you read FRL for the book reviews, you may want to pass this up.

I’ve had an idea for MPOW percolating for a very long time and recently decided to blog it, but embargoed this post for a while. We did usability testing this week for the next version of MPOW, and I didn’t want any of the subjects to read this, just in case they stumbled across my blog. (Oh, and I did Google the subjects. ;> )

Basically this migration is focused on everything except the search function for the site. That will sound odd, given that MPOW is a searchable web portal, but we had pressing sustainability and management issues with our old design and software, and needed to move to a database-driven structure sooner than later. Plus, for reasons addressed below, search will not be an easy or very cheap capability to fix.

The roll-out will be an improvement over what we have, but–like moving MPOW off a donated server and bandwidth and going to commercial hosting–it’s more of an infrastructure change; the actual site won’t be that different. There are some new features users will like, such as the ability to email an item, breadcrumbs within topics, better browse overall, and a generally more modern and polished look and feel, but a lot of what we are doing is related to sustainability and management. It doesn’t address our search function, which is, in a sense, broken.

Just what is wrong with MPOW’s search function? Nothing, if it’s 1990 and you’ve never searched anything other than an OPAC. LII’s search function faithfully matches against the metadata in the entries we write to describe Websites. Just plug in nice broad book-based terms such as “movie reviews,” and you’ll find good stuff.

However, try to search MPOW the way users really search, leveraging the content of the Web, and you won’t be so lucky. You have to think like a librarian–a librarian of an earlier era–to translate “Mel Gibson” to movie reviews, or “chocolate cupcakes” to recipes or baking. As search portals go, it’s not much more than a well-annotated gopher with an increasingly nice Website wrapped around it, or a very limited OPAC (if that isn’t redundant).

(On the other hand, MPOW is an unqualified success on other terms, particularly as one of the world’s best Website announcement services–a capability I have promoted heavily in part because we do this so well.)

I’ve heard librarians explain to me that the value of MPOW is that they can use it to train the public how to use a subject directory. While MPOW has many fans who are willing to put up with its peccadillos, to hear it used as an example of The Last of Its Tribe sends chills down my spine (and I do not mean frissons of pleasure).

I knew MPOW had this fundamental problem lo, even before I came on board in the fall of 2001. I never used LII before I became its manager. I admired it in principle, I recommended it, I even wrote about it, but I didn’t use it myself. I felt I could get everything I need from search engines, and had the skills to discern the good from bad quickly enough to make it worth my while.

My first thoughts about fixing the vexing search problem with LII were shelved when the California budget tanked shortly after I arrived and LII went into shaky survival mode. Just existing seemed like a miracle; after two budget cuts, one within a budget year, survival meant running in place, with very tiny prinks to our content organization on our main site to consolidate it Yahoo-style from 70 topics into just over a dozen main categories. (To see the olden site, visit MPOW in the Wayback Machine.)

I often think LII exists today because I am often too dense to understand what I’m hearing. I remember at one point a board member saying, “Too bad about LII, Karen–you did your best.” I about blew up! LII wasn’t going anywhere just yet, I replied. It ain’t over til it’s over! But somehow, with a lot of support from good people, we survived. It helps that for a statewide service we run on a shoestring and have very little overhead (we all work from home), but still, I thank my lucky stars I am dim-witted enough not to read the handwriting on the wall even when it is six feet high.

When our budget stabilized, I looked at some of the other directions we could go into. Infomine, an academic portal I find interesting, has a fascinating capability to scrape valuable metadata into records from Web sites added to its portal. (It also automatically adds robot-generated sites, but that’s a story for another day.) I think that’s a great idea for enhancing search. However, adding 100 key terms to metadata will help broad-term searching, but it doesn’t emulate the more specific searches people do in Google.