Revolutionary Testing Tools for AJAX Applications
http://blog.browsermob.com/2008/12/faq-how-do-you-compare-to-selenium/
A few years ago, we used JMeter to try and load test a chat application we were building. Since the chat application used Comet and AJAX, I knew that the JMeter test was going to be grossly inaccurate. However, we did the best we could with tools that clearly weren't right for the job, as nothing was out there that could simulate an AJAX or Comet load test.
A few weeks ago, while working with some other consultants on a project, we discussed the use of Selenium to run some functional testing on a Firefox Extension that inserts signatures into an email. We needed a tool that could automatically send an email using Gmail, Yahoo, and Hotmail, and verify that the signature was still injecting. From my days writing HTML Reply Signatures for Gmail, I knew that this was indeed important, as any change by these third parties could have a drastic effect on the tool's ability to insert a signature.
Today, one consultant and I worked to try to insert an email address into the "To" field of the Gmail Compose window using Selenium. I've done this already in the Firefox extension, and anyone who has ever written a Greasemonkey script or Firefox extension that works with Gmail 2 can attest that this is not always an easy task if you are working outside of the API exposed by Gmail. Google engineers have put lots of effort in writing server-side code that ensures that all the id attributes in the Gmail application randomly change each session. Their creativity has sparked our creativity, and we have solved this problem. With enough time and patience, any problem can be solved.
Anyway, with Selenium, one must locate elements in the HTML of a web application by using XPATH queries. Armed with my trusty Firebug tool, I inspected the element in question and worked my way up the tree one element at a time, writing down the element tag names and positions at each step. Like a complex code, we ended up with long XPATH queries, a seemingly endless series of div[1]/div[2]/div[1]/div[1]. Although I would have preferred to just use JavaScript and a good for loop or regular expression to locate what I wanted, being a code cracker and approaching this problem from another angle was an eye-opener, XPATH is perhaps the way to go!
So that's a little background on my experience so far with testing tools. Selenium isn't a load testing tool of course, but it's a great tool to automate functional testing. It's almost like a macro recorder, similar to iMacros. Our plan is to set our test up on a VMWare image and configure this to run automatically every day. We'll get emails with our signature if it's successful, and broken or no email if it's not successful. We're also configuring it to be really loud and obnoxious if it does detect that something is amiss, just so we don't gloss over the automated emails.
Selenium can do functional testing, but what about load testing? Sure, Selenium can do load testing as well! Selenium uses real web browsers in testing, but it's up to the person doing the tests to go out and purchase the hardware to run the tests. In other words, if you want to do a 100 user load test with Selenium, you need to have 100 computers in your lab and 100 people to all click "Run" at the same time.
This is analogous to a symphony with you as the conductor who tells everyone exactly when to click "Run Test".
Of course, coordinating such an event can be costly, so one of the developers of Selenium Remote Control founded BrowserMob. BrowserMob solves this issue by providing the hardware. They claim to load test your web application using Selenium scripts and N number of real web browsers! I would be really interested to know how this was accomplished and how the costs have been kept so low.
To start, they probably use VMWare and Linux, which is one way to reduce the amount of physical hardware while still trying to simulate real hardware. They say they use Firefox 3.0, which is available on Linux. Finally, if Selenium Remote Control launches Firefox browsers to simulate functional tests, then my guess is BrowserMob is controlling a series of Selenium Remote Control instances, all running on VMWare Linux machines, and allowing BrowserMob to monitor the load.
While there are some costs associated with BrowserMob, I imagine one could use Selenium Remote Control installed on several VMWare images to run a small load test. If anyone has experience with either Selenium Remote Control or BrowserMob, I'd love to hear your thoughts on the two products!
