On Wed, 15 Dec 2004 14:28:51 -0500, Clay Redding <[log in to unmask]> wrote:
> Hi Eric,
>
> Not necessarily. If you're up for trying PostgreSQL, their XML
> functionality works *really* well.
>
> http://www.throwingbeans.org/tech/postgresql_and_xml.html
[...]
> Eric Lease Morgan wrote:
>
> > Again, thank you for the prompt reply.
> >
> > Actually, the implementation you suggest was the path I was
> > considering. Unfortunately, this means leaving some sort of text lying
> > around on my file system for importing. Similarly, it poses the problem
> > of editing; in order to edit data in such an implementation I will need
> > to export the big chunk, edit, and re-import. That is sort of klunky,
> > but still it is what I was considering.
Let me also mention eXist[1] as an XML database that supports XPath
and XQuery. Throwing Beans mentions some of eXist's advantages in a
posting from September [2].
In my mind, searching large swaths of full-text is rather different
than searching structured metadata, especially on controlled
vocabulary index points. One idea that comes to mind is using either
the Throwing Beans/PostgreSQL or eXist solutions for storage and
XPath-based access, and creating full-text indices with Lucene[3]. (I
mention Lucene because I recall reading a few months back that eXist
and Sleepycat were optimized for XPath queries and so might not
perform adequately in full-text searches.) In developing a Lucene
index of a corpus of TEI texts, I suspect that for each indexed chunk
(chapter, section, paragraph) you could include the XPath expression
pointing to that chunk as a thing that could be returned. Do
full-text searches against Lucene, but pull actual text to be
transformed from eXist, using XPath expressions returned from Lucene.
Browsable representations of document structures would, I think, also
be easier to get from eXist or PgSQL/Throwing Beans than from Lucene.
You could probably just store the documents in eXist, and spit them
out temporarily to let Lucene index them. I am not sure that Lucene
would permit you to selectively re-index corrected documents, though.
You might have spit out and re-index all the docs after posting a
corrected version back to eXist.
I suspect that a system like this could also be made to work for
collections of EAD finding aids, though perhaps they have less need of
heavy-duty full-text searching than literary or philosophical texts.
Just a few thoroughly untried ideas!
Chuck
[1] <http://exist.sourceforge.net/>
[2] <http://www.throwingbeans.org/tech/xml_databases_with_exist_and_coldfusion.html#000048>
[3] <http://jakarta.apache.org/lucene/docs/index.html>
|