 |
 |
I've been out, and so haven't been following this, but I'll try to explain what xISBN is doing. The xISBN service runs from a fairly simple file. We generate clusters of ISBNs based on FRBR work-set groupings and refine them so that each ISBN is in only a single cluster. Each ISBN in each cluster of size 2 or more was written to a file, followed by the rest of the ISBNs in that cluster, sorted by number of library holdings. That file is sorted by ISBN, and we currently actually read that file into memory as one big string. We search the string to find the proper group using some simple indexes and binary search. Since all the information is in memory and all that has to be done is to reformat it into XML, the service is very fast (but sits there with several hundred megabytes of data). Hope that's a clear enough explanation of how we're currently doing it. The main computation is the refinement of the initial clusters. It takes around an hour to run on our 48-cpu cluster, about as long as the initial FRBR grouping on WorldCat. --Th -----Original Message----- From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of Jonathan Gorman Sent: Tuesday, May 23, 2006 9:54 AM To: [log in to unmask] Subject: Re: [CODE4LIB] Musings on using xISBN in our Horizon catalog > > That's why I'd love to know whether the xISBN database uses a common > identifier for each set of ISBNs, and whether (and I know 'pretty > please' is a poor justification for changing an API) it might be exposed > for this reason. > Hopefully the OCLC people can answer that. It might be in the work Andy suggested yesterday. One idea I had while yesterday was if you don't care that much about the id internally you could use an auto-increment. To clarify, we'll assume that any isbn in a set will return the same set in xISBN. IE asking for isbns related to a returns a,b and c. Asking for b or c should return a,b,c. So we can do as Andy suggested and start building our table by taking the set of all current isbns, normalized a bit I'd imagine. In a computationally-expensive method: Start with the first isbn (x) and get the set of isbns from xISBN that is related (A). Iterate over every member of A testing for the following: is the member assigned to a group already. If it has, stop the loop and assign x to the same group. If none in A have been assigned a group, start a new group and add x. You'll have to do this every once in a while to make sure you're getting all the new books. Hopefully this makes up for the advice I gave yesterday ;). I'm sure you can probably come up with a better algorithm though, something about the backward-lookup everytime makes me think that there's a better way. ps. Andy's right, normalization is a good, good thing. Only reason I suggested looking at the costs was I was thinking it would be a lot easier than trying to come up with a method to generate unique ids for a "group" since my grasp of FRBR/xISBN is a little shaky I'll avoid any specific terminology. (Like I said in my original email, having a identifier or groups is a definite advangtage).