 |
 |
On Jan 19, 2007, at 6:53 AM, Erik Hatcher wrote: > Great information. I apologize for being a late comer to the game > and bringing up FAQs. Well, I'm not certain Q is A-ed very F. > What about date normalization? > > One thing that must be considered when doing faceted browsing is that > it works best with some pre-processed data, such as years rather than > full dates. The question becomes where does the logic for stripping > out the years belong? Solr could do it if configured with a custom > analyzer for certain fields, or the client could do it. Is there > XSLT to do this sort of thing with dates available? I think the XSLT side of this has been hashed out. A couple questions that come to mind on the side of the underlying search engine. For simple dates, say plain years, straight text matching is okay. But when the record spans some time interval, say 1940s or 1776-1783 or 17th century, it would be nice to have some support from the underlying search engine. Then maybe the occasional user wants to express a query in an inteval. I've done the "list every year in the interval" and it works if everything is a year, but I find it a bit unsatisfying. The kind of date operations that you find in RDBMSes seem to be useful. I'm not certain what Luncene offers. But for facets, I guess a list of discrete years is just what you're likely to have to work with. displaying the year facet and offering a range could be interesting. Apologies for rambling. I woke up a long time ago in another time zone. -Tod > On Jan 19, 2007, at 5:58 AM, Tod Olson wrote: > >> On Jan 19, 2007, at 4:07 AM, Erik Hatcher wrote: >> >>> On Jan 17, 2007, at 3:26 PM, Andrew Nagy wrote: >>>> One thing I am hoping that can come out of the preconference is a >>>> standard XSLT doc. I sat down with my metadata librarian to >>>> develop our >>>> XSLT doc -- determining what fields are to be searchable what >>>> fields >>>> should be left out to help speed up results, etc. >>>> >>>> It's pretty easy, I think you will be amazed how fast you can >>>> have a >>>> functioning system with very little effort. >>> >>> You're quite right with that last statement. >>> >>> I am, however, skeptical of a purely MARC -> XSLT -> Solr solution. >>> The MARC data I've seen requires some basic cleanup (removing >>> dots at >>> the end of subjects, normalizing dates, etc) in order to be >>> useful as >>> facets. While XSLT is powerful, this type of data manipulation is >>> better (IMO) done with scripting languages that allow for easy >>> tweaking in a succinct way. I'm sure XSLT could do everything that >>> you'd want done; you can also drive screws in with a hammer :) >> >> So the punctuation stripping has already been done in XSLT. >> >> LoC has a MARCXML -> MODS XSLT stylesheet [1] which strips out the >> evil >> ISBD punctuation. I've generally found mapping from MODS to be more >> convenient than mapping from MARC, so while it's an extra step, it >> does >> save a little programmer time since some of the hidden hierarchy in >> the >> MARC data is made explicit in the MODS structure. >> >> If hopping through MODS is unacceptable, the LoC has the punctuation- >> stripping nicely tucked away into a MARC Conversion Utility >> Stylesheet >> that you could use directly in a MARC XML -> Solr transformation. [2] >> >> [1] http://www.loc.gov/standards/mods/v3/MARC21slim2MODS.xsl >> [2] http://www.loc.gov/marcxml/xslt/MARC21slimUtils.xsl >> >> >> Tod Olson <[log in to unmask]> >> Programmer/Analyst >> University of Chicago Library