 |
 |
On May 9, 2008, at 1:42 PM, Jonathan Rochkind wrote: > The Blacklight code is not currently using XML or XSLT. It's indexing > binary MARC files. I don't know it's speed, but I hear it's pretty > fast. Right, I'm talking about the java indexer we're working on, which we're hoping to turn into a plugin contrib module for solr. It processes binary marc files. We're getting times of about 150 records / second, but that's on an unfortunately throttled server and we're munging each record significantly (replacing musical instrument and language codes with their English language equivalents, calculating composition era, etc). Casey, you say you're getting indexing times of 1000 records / second? That's amazing! I really have to take a closer look at MarcThing. Could pymarc really be that much faster than marc4j? Or are we comparing apples to oranges since we haven't normalized for the kinds of mapping we're doing and the hardware it's running on? Bess Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 [log in to unmask] (434) 243-2305