OK, my last reply to a reply today, I swear, for this list at least :-)

>> * the indexer allows control of how the data is normalized
> Is this the indexers job?  I say not.

I probably should have said that "the indexer does not preclude special
normalization", and for that matter, I probably should not have used
"normalized" as a descriptive term at all, since it brings in all sorts of
semantic baggage that is not related. By normalize, I mean controlling how
stemming, and things like accents and special characters are handled, and
by being helpful to this process, I mean that the indexer is flexible, not
that it should necessarily step in and do all this work. There's a really
interesting presentation from Apachecon 2005 about the Center for Natural
Language Processing's (CNLP) work with Lucene
<http://www.cnlp.org/apachecon2005/>, and there is some description of how
they handle multiple languages, so what I probably really meant is
"supporting more of what CNLP is doing", but that would have implied I
understood it much better than I do. Anyway, Lucene does brilliantly on
all this.

>The better granularity to consider is the interface
>to the index, like SRU or Solr's custom interface, etc.  And the
>library world already has these standards in place that could easily
>be put on top of Lucene or Solr.

Well, sure, that's good too, but if I wanted to limit searches to journals
the library subscribes to in, say, the 7 million articles in the Scopus
dataset, I could see combining a Lucene index of that content, or at least
the holding identification part of that content, with a library index, and
then limiting searches to content that is available, or not available, or
both, and so on. In another life-time, I used to work with a system called
SPIRES, which stored indexes in their own databases, and these kind of
combinations were possible even when the Berlin wall was still standing.
The library world has standards for identifying the availability of
individual items, and these rock, but I don't think there is that much for
collection scoping. Of course, if I could get the Scopus content and such
directly, this probably wouldn't matter, but I really wonder if there's
some index construct that could possibly be sharable among all content
providers without round-tripping to check on the status of each item.

Of course, there are probably political/legal issues in all this that make
the technology involved seem trivial...

art