For what it's worth, the VuFind community has recently been playing with Selenium (not an especially new or exciting technology, I realize... and probably one of the things you were thinking of for approach #1). The good news is that it plays well with Hudson, and we have been able to get it to successfully automatically test AJAXy code in Firefox as part of our continuous integration process. The bad news is that it's incredibly slow -- that successful test takes ten minutes to execute, and all it does is load one web page and confirm that a lightbox opens when a button is clicked. I wouldn't realistically expect this sort of thing to be FAST, but the current performance we are experiencing stretches belief a bit -- we're still investigating to see if we're doing something wrong that can be improved, but the general consensus seems to be that Selenium is just really slow on certain platforms. It's a shame, because I think we could potentially write a very comprehensive and powerful test suite with Selenium... but tests are significantly less valuable if they can't give you reasonably quick feedback while you're in the midst of coding!
In any case, I'm happy to share my limited experience with Selenium if it's of any use (some VuFind-specific notes are here: http://vufind.org/wiki/unit_tests#selenium and more can probably be gleaned by looking at VuFind's test-related configuration and scripts). I'd also be very interested to hear if anyone has overcome the speed problems (which I've encountered under both RedHat and Ubuntu, possibly related to using a virtual frame buffer) or if there is a better, equivalent solution.
> -----Original Message-----
> From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of
> Jonathan Rochkind
> Sent: Wednesday, January 12, 2011 11:32 AM
> To: [log in to unmask]
> As far as I can tell, while there are several, there are none that are
> actually Just Work good. It seems to be an area still in flux, people
> coming up with an open source way to do that that is reliable and easy
> to use and just works.
> The main division in current approaches seems to be between: 1) Trying
> to automate _actual browsers_ so you know you've tested it in the real
> browsers you care about (the headaches of this are obvious, but people
> testing idiosyncracies of particular browsers (I would lean towards
> one myself, I'm willing to give up what it gives up for something that
> works a lot simpler with less headaches).
> On 1/11/2011 7:21 PM, Bess Sadler wrote:
> know we need to test the js portions of our applications, but we
> haven't settled on a tool for that yet. I've heard good things about
> celerity (http://celerity.rubyforge.org/) but I believe it only works
> with jruby, which has been a barrier to getting started with it so far.
> in a way they like? Feel like sharing?
> > Thanks!
> > Bess