Just a follow-up from my mail yesterday, I received a few messages that
suggest I should have made some aspects more clear. One is that the
parallel setup was a very quick test, I tried putting a dozen targets in
the same setup but to really nail down the extent of the added efficiencies
of concurrent retrievals, you would have to work around browser caching and
probably lots of other factors. I also don't have any exception clauses
built into the generator, so if one target times out, you get the beauty of
an unhelpful message and options for a java stacktrace. So, yes, it does
break if the network is slow. Timeouts must be a fairly intense kind of
calculation for federated searching, I guess if you were trying to do some
sort of de-duping you would want to try to wait for every target but
there's got to be some tough trade-offs here.

As near as I can tell, the parallel processing is for collecting the
content, the aggregation piece in Cocoon seems to have to hear back from
all of the targets before pushing any content further down the pipeline.
You could set timeouts in the generator, but I think Cocoon would still be
somewhat at the mercy of one unresponsive target. One part of federated
searching that probably shouldn't be underestimated is the browser itself.
If you can push some of the content out through some sort of iFrame setup,
for example, the browser still shows the content surrounding the iFrames
while doing a pretty good job of juggling multiple processes. It's the same
as when your web page has a sequence of <img src=".."> elements, a user
still gets some content right away and the images get added as they are
available or as they can be processed.

art