The Gordian Knot

David Pattern’s jetlag-fueled creativity binge continued last week, even after coming up with the marvelous tag cloud browsing interface we discussed earlier. His encore? An Amazon/Netflix-style addition to the full bib in HIP that informed the user, “People who borrowed this, also borrowed…” and listed up to 10 other suggested bib records (see the bottom of this screenshot). Wow — how’d he do that? Well, he’s using the circ_tran table and his mighty Perl skills to create the recommendation lists on a periodic basis (it’s not a realtime thing yet).

It’s an awesome idea — but one that brings up an issue for those of us in the shadow of the USA PATRIOT Act. Many of us drop our circ transaction records like a hot potato as soon as the items are returned and the blocks are cleared. I raised the issue in a comment to David’s post — would there be a way to gather “co-circulation” information (things borrowed by the same borrower) while somehow not leaving a link that can be followed back to a specific borrower record? In those comments, David says we could separate circ transactions from their borrowers and still produce a list of “people who borrowed this, also borrowed these at the same time” — which might still have some value but wouldn’t be nearly as rich as what he’s providing in his prototype. And then Casey (from Seattle) proposed a couple of fascinating ideas involving encrypting borrower ID numbers — my favorite involves asking some trusted third party outside the US to encrypt the borrower numbers using an encryption key that only they would know, then return the “laundered” records to us with borrower numbers that were encrypted in such a way that a given borrower’s encrypted number was always the same, but could not be decrypted and traced back to the original record without cooperation from the overseas party. I wonder if there are any SirsiDynix libraries outside the US that would be interested in performing that type of service for us here?

Davey and Casey conclude that it might be easier to just use Amazon Web Services, which then leads to an interesting question: how close would a list of titles that “People who bought this at Amazon also bought…” match a list of titles that “People who borrowed this at the library also borrowed…”? I wonder if there might be still another method for libraries to do this cooperatively. What if we were to build a large cooperative database of anonymized circ_tran data? Could we do it in a way that the anonymized borrower ID in the shared table would be utterly untraceable back to our libraries, yet each time we upload, each borrower’s new borrowing activity would be associated with the correct anonymized borrower ID in the shared table? That way we could potentially develop an even richer datastore, and even richer recommendations.

Whatever strategies we use, an article in this morning’s New York Times suggests to me that this is far from an idle intellectual exercise — solutions like this are going to be increasingly in demand. This is one of those places where highly-social “Library 2.0″ ideals — so much discussed in the last month or so on a number of library blogs — are destined to go crashing headlong into some ugly reality, unless we can find some practical (and hopefully elegant) workarounds. Fortunately, this little online brainstorming with Dave and Casey makes me think that we can definitely pull it off.