 |
 |
I think there are many good reasons why XSLT is absolutely the wrong tool for the job of indexing MARC records for Solr. 1) Performance/Speed: In my experience even just transforming from MARCXML to MODS takes a second or two (using the LoC stylesheet), due to the stylesheet's complexity and inefficiency of doing heavy-duty string manipulation in XSL. That means you're looking at an indexing speed of around 1 record/second. If you've got 1,000,000 bib records, it'll take a couple of weeks just to index your data. For comparison, the indexer of our commercial OPAC does about 50 records per second (~6 hours for a million records) and the one I've written in Jython (by no means the fastest language out there) that doesn't use XSL can do about 150 records a second (about 2 hours for 1 million records). 2) Reusability: What if you want to change how a field is indexed? You would have to edit the XSLT directly (or have the XSL stylesheet automatically generated based on settings stored elsewhere). a) Users of the indexer shouldn't have to actually mess with programming logic to change how it indexes. You shouldn't have to know a thing about programming to change the setup of an index. b) It should be easy for an external application to know how your indexes have been built. This would be very difficult with an XSL stylesheet. Burying configuration inside of programming logic is a bad idea. c) The Solr schema should be automatically generated from your index setup so all your index configuration is in one place. I guess you could write *another* XSL stylesheet that would transform your indexing stylesheet into the Solr schema file, but that seems ridiculous. d) Automatic code generation is evil. Blanchard's law: "Systems that require code generation lack sufficient power to tackle the problem at hand." If you find yourself considering automatic code generation, you should instead be considering a more dynamic programming language. 3) Ease of programming. a) Heavy-duty string manipulation is a pain in pure XSLT. To index MARC records have to do normalization on dates and names and you probably want to do some translation between MARC codes and their meanings (for the audience & language codes, for instance). Is it doable? Yes, especially if you use XSL extension functions. But if you're going to have huge chunks of your logic buried in extension functions, why not go whole hog and do it all outside of XSLT, instead of having half your programming logic in an extension function and half in the XSLT itself? b) Using XSLT makes object-oriented programming with your data harder. Your indexer should be able to give you a nice object representation of a record (so you can use that object representation within other code). If you go the XSLT route, you'd have to parse the MARC record, transform it to your Solr record XML format, then parse that XML and map the XML to an object. If you avoid XSLT, you just parse the MARC record and transform it to an object programmatically (with the object having a method to print itself out as a Solr XML record). Honestly, all this talk of using XSLT for indexing MARC records reminds me of that guy who rode across the United States on a riding lawnmower. I am looking forward to there being a standard, well-tested MARC record indexer for Solr (and would be excited to contribute to such a project), but I don't think that XSL is the right tool to use. --Casey