Ian Bicking: the old part of his blog

Re: Web application acceptance testing

I would note that the kind of testing you're referring to (presumably making sure nothing is broken) is not what one might think of as "acceptance testing". To me, after you write the program, acceptance testing is when the user makes sure the program is fast enough, usable enough, and works right to solve the problem, which is larger in scope than automated testing to find bugs.

At my work, we've considered this sort of automated testing for some time, but there is one increasingly important part of web development that they almost completely ignore: JavaScript. What would be nice is a Firefox plugin for acceptance testing that works with the JavaScript interpreter -- almost like an old-school macro recorder. Unfortunately, this brings up the issue of testing between browsers.

Ultimately for my money on normal-size apps, I think there's no true replacement for spending a few hours each release trying everything on your site off a todo list. We do that each with separate browsers and usually end up catching most problems.

-Ken

Comment on Web application acceptance testing
by Ken Kinder

Comments:

True... I tend to use "acceptance testing" for everything that isn't "unit testing", but that's probably not very accurate. Functional testing or system testing is probably a better term. Though Grig pointed me to this: http://atomicobject.com/systir.page (a Ruby test system) which is really an acceptance test (specifying requirements at a fairly low but not obscenely low level), and that could fit into this model. It's not a comprehensive acceptance test, but it isn't too far off. Automated acceptance testing is possible -- FIT is an example of that sort of thing.

Selenium or Pamie are options for Javascript-run tests (from Grig's article). John Lee also put Javascript in his to-do for his stack (ClientForm, ClientCookie, Mechanize, etc); I think using Mozilla's interpreter or something. But who knows about that...

Generally I do manual Javascript testing, but that still leaves a place for automated tests. By avoiding dynamic Javascript and making sure your app degrades properly in the absence of Javascript, you can avoid regressions in your Javascript and make the application accessible to an automated testing tool.

Also, by doing this as a script you can avoid sensibility entirely -- you can jump to another page based on any calculation you choose, possibly derived from a regular expression-based parsing or whatever works. You aren't restricted to just following links and filling in forms that you can find on the page.

# Ian Bicking