 |
 |
Eric this is awesome. It would probably be worthwhile to document all this somewhere (outside this mailing list), maybe the code4lib wiki? -Ross. On Tue, Jun 17, 2008 at 4:29 PM, Eric Lease Morgan <[log in to unmask]> wrote: > > On Jun 17, 2008, at 12:06 PM, siznax wrote: > >> the Zebralist appears to be relatively inactive. >> anyone here have experience indexing MARC binaries >> with zebra? > > > I will try to outline here how to index (and search) MARC records using > Zebra, but tweaking the indexing process is a bit trickier than I know how > to do. > > 1. Install yaz, zebra, and all of their friends. I have found that the > "standard" make process works pretty well, but allow yaz and zebra to > specify where it puts various configuration files. The extra specification > is not worth the effort. > > 2. Save your MARC records someplace on your file system. By "binary" MARC > records, I suppose you mean "real" MARC records -- MARC records in > communications format -- MARC records as the types of records fed to > traditional integrated library systems. This is opposed to some flavor of > XML or "tagged format" often used for display. > > 3. Create a zebra.cfg file, and have it look something like this: > > # global paths > profilePath: .:./etc:/usr/local/share/idzebra-2.0/tab > modulePath: /usr/local/lib/idzebra-2.0/modules > > # turn ranking on > rank: rank-1 > > # define a database of marc records called opac > opac.database: opac > opac.recordtype: grs.marcxml.marc21 > attset: bib1.att > attset: explain.att > > 4. Index your MARC records with the following command. You should see lot's > of great stuff sent to STDOUT. > > zebraidx -g opac update <path to MARC records> > > > You have now created your index. Once you get this far with indexing, you > will want to tweak various .abs files (I think) to enhance the indexing > process. This particular thing is not my forte. It seems like black magic to > most of us. This is not a Zebra-specific problem; this is a problem with > Z39.50. > > Next, you need to implement the client/server end of things: > > 5. Start your server. This will be a Z39.50 server -- a "kewl" > library-centric protocol that existed before the Internet got hot: > > zebrasrv localhost:9999 & > > 6. Use yaz-client to search your index: > > & yaz-client > Z> open localhost:9999/opac > Z> find origami > Z> show 1 > Z> quit > > Using the yaz-client almost requires a knowledge of Z39.50. Attached should > be a Perl script that allows you to search your server in a bit more > user-friendly way. To use it you will need to install a few Perl modules and > then edit the constant called DATABASE. > > Even though Z39.50 is/was "kewl" it is still pretty icky. SRU is better -- > definitely a step in the right direction, and Zebra supports SRU out of the > box. [1] > > 7. Create an an SRU configuration file looking something like this: > > <yazgfs> > <server> > <config>zebra.cfg</config> > <cql2rpn>pqf.properties</cql2rpn> > </server> > </yazgfs> > > 8. Acquire a "better" pqf.properties file. PQF is about querying Z39.50 > databases. It is ugly. It was designed in a non-Internet world. Instead of > knowing that 1=4 means search the title field, you want to simply search the > title. Attached is a "better" pqf.properties file, and it is "better" > because it maps things like 1=4 to Dublin Core equivalents. Save it in a > directory called etc in the same directory as your zebra.cfg file. (Notice > how the zebra.cfg file, above, denotes etc as being in zebra's path.) > > 9. Kill your presently running Z39.50 server. > > 10. Start up a SRU server: > > zebrasrv -f sru.cfg localhost:9999 & > > 11. Use your HTTP client to search the SRU server. Queries will look like > this (with carriage returns added for readability): > > http://localhost:9999/opac? > operation=searchRetrieve& > version=1.1& > query=origami& > maximumRecords=5 > > The result should be a stream of XML ready for XSLT processing. > > All of the above is almost exactly what I did to create an index of MARC > records harvested from the Library of Congress and the University of > Michigan's OAI data repository (MBooks). [2] Take a look at the HTML source. > Notice how the client in this regard is only one HTML file containing a > form, one CSS file for style, and one XSL file for XML to HTML > transformation. > > HTH. > > [1] SRU - http://www.loc.gov/standards/sru/ > [2] Example SRU interface - http://infomotions.com/ii/ > > -- > Eric Lease Morgan > Head, Digital Access and Information Architecture Department > Hesburgh Libraries, University of Notre Dame > > (574) 631-8604 > > > > > > > > > >