 |
 |
Those of us involved in the Blacklight and VuFind projects are spending lots of time recently thinking about marc records indexing. We're about to start running some performance tests, and we want to create unit tests for our marc to solr indexer, and also people wanting to download and play with the software need to have easy access to a small but representative set of marc records that they can play with. According to the combined brainstorming of Jonathan Rochkind and myself, the ideal record set should: 1. contain about 10k records, enough to really see the features, but small enough that you could index it in a few minutes on a typical desktop 2. contain a distribution of kinds of records, e.g., books, CDs, musical scores, DVDs, special collection items, etc. 3. contain a distribution of languages, so we can test unicode handling 4. contain holdings information in addition to bib records 5. contain a distribution of typical errors one might encounter with marc records in the wild It seems to me that the set that Casey donated to Open Library (http://www.archive.org/details/marc_records_scriblio_net) would be a good place from which to draw records, because although IANAL, this seems to sidestep any legal hurdles. I'd also love to see the ability for the community to contribute test cases. Assuming such a set doesn't exist already (see my question below) this seems like the ideal sort of project for code4lib to host, too. Since code4lib is my lazyweb, I'm asking you: 1. Does something like this exist already and I just don't know about it? 2. If not, do you have suggestions on how to go about making such a data set? I have some ideas on how to do it bit by bit, and we have a certain small set of records that we're already using for testing, but maybe there's a better method that I don't know about? 3. Are there features missing from the above list that would make this more useful? Thoughts? Comments? Thanks! Bess Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 [log in to unmask] (434) 243-2305