> When I tackled Ebsco, I ran into issues of site authentication via
> cookies that were passed to the search gateway but not on to the client
> browser. Peter Binkley, at the University of Alberta recommended a proxy
> configuration to balance off this issue.  Essentially those connections
> would have to continue to operate inside a search gateway proxied session.

yeah...haven't really tackled passing the searches thru...that's why i
just tried to give the stable urls from the result lists if i could get
them....something to consider tho

> I don't know how the perl tools stack up in terms of parallel search
> streams.  The php/curl combination is purely serial and the last targets
> will time out if there is a tardy responder in the middle of the serial
> queue.

yeah....another reason for some of my decisions, esp. the iframe stuff;
not terribly pleased with it, but avoided the concurrency issue...

there is an extension to perl's LWP that allows parallel searching:

    http://www2.inf.ethz.ch/~langhein/ParallelUA/

haven't looked very closely, but would probably be another solution....

my main problem right now is the parsing....ugh, ugh...ugh....

> Art Rhyno, at the University of Windsor, suggested a parallel
> approach might be possible in a Cocoon environment.  This has the
> advantage of passing all the inbound HTML pages through JTidy and giving
> you the XHTML/XML compliant input stream you wanted (in most cases, even
> when the output from the target was some distance from compliance).

another possible solution....art's done some pretty cool things with
cocoon, but i haven't tried that kool-aid yet ;)