Tools for Putting Web Apps to the Cross-Browser Test
Cross-browser testing for both security and functionality issues is a crucial step for any serious Web application. There are several automated tools out there for assisting the task, including some from open source projects like Selenium. However, several problems remain in cross-platform testing procedures, even with proven tools.
Feb 26, 2010 5:00 AM PT
Web-based applications and cloud computing have presented new challenges for software developers. Most software makers are by no means tone-deaf to user concerns about security and usability issues, but even those software writers who are receptive to these worries must contend with hard-to-plug holes that can open up in cross-platform programs such as Web browsers.
For Web app developers, the problems occur on two fronts. Not only do they have to harden the application itself, but they also have to keep up with the occasional new browser release -- updates to Microsoft's Internet Explorer, for instance, as well as frequent version upgrades for Apple's Safari, Mozilla's Firefox, Opera, Google Chrome and others.
With more applications built for the Web, cross-browser testing is crucial to application performance. Numerous automated helpers are available from a variety of commercial and community-based software testing tools. For example, over 2 million users have turned to the Selenium Project's open source, cross-browser testing platform to solve the various security and functionality flaws that crop up in software code. Selenium is currently in use at companies such as Google, Yahoo, eBay and Salesforce.com.
However, several problems remain in cross-platform testing procedures, even with proven tools. Users often require commercial support for Selenium. Others need enhanced features and speed. Other enterprises simply lack the infrastructure for doing their own sophisticated product testing.
"We address the routine that each developer faces. It is laborious to make sure that all the patches are current and that the latest software version is installed. Sauce Labs does that for the developers so that problem goes away for them. Developers traditionally struggled in their labs with single versions of a software application," John Dunham, CEO of Sauce Labs, told LinuxInsider.
Barn Door Theory
Failure to perform cross-platform tests is unthinkable today. Code writers cannot rely on defined protocols and interfaces.
No two systems are alike, and this holds true for platforms, whether it's various hardware manufacturers or operating system platforms. The underlying operating systems and their code bases differ. "Because of these differences, it is very important that developers conduct due diligence on their products before releasing to the general public," Ken Pappas, president of True North Security, told LinuxInsider.
Hackers used to pay close attention to network breaches via network vulnerabilities. Now they've largely shifted their focus to application vulnerabilities due to developers not doing a good enough job of testing their products for security vulnerabilities on cross platforms. Good developers will test and certify their products on multiple platforms to protect against hackers, he said.
Cross platform-testing is an absolute must, advised Mandeep Khera, CMO for Web application security vendor Cenzic. However, it should be balanced with the developer's resources and timelines.
"Ask where are most of your users. Use the 80/20 rule to decide how much and when and repeat if it works. If resource and time are not issues, go full-force with all platforms," Khera told LinuxInsider.
More Than Security
The Selenium Project did not originate with a quest to make cross-platform software more secure, noted Dunham. Software testing is done for general functionality more than security issues. The developers sought a solution to make it easier for them to make cross-platform apps like the ones used in SaaS (Software as a Service) and cloud-delivered software more reliable.
"The browser wars were getting underway. Software developers needed to support multiple browsers," Jason Huggins, cofounder of Sauce Labs, told LinuxInsider.
The big problem in developing Selenium was that features would work in one browser, like IE but not in Firefox. Huggins would fix one, and then it would break in the other.
The Selenium Project interacts with a Web site the way a user would. The software goes to the site, clicks on an image, enters some text, clicks on a button -- all the things typical users do. That's what was built into Selenium, according to Huggins.
Selenium is a testing environment for developers. It's a robotic platform for which developers can write scripts. They can remotely control the browser to have it do certain functions and then compare the results with a pass-fail value and make a judgment based on that, he explained.
The software is open source. What Sauce Labs does is apply the factors for its use with the clouds.
"All of those so-called air traffic control factors are what Sauce Labs adds to Selenium," Dunham said.
Sauce Labs provides peer support and value-added packages. That's the open source business model that can lead to a steady flow of money as a commercial operation.
So far, Selenium has had over 2.6 million downloads by QA testes and developers. What Sauce Labs is doing is positioning itself as the Red Hat for Selenium, Dunham said.
His reference to commercial enterprise Red Hat hinted at what Dunham's goal is for Sauce Labs. Red Hat Linux is a commercial distribution of the free community-based Linux operating system the Red Hat company develops as the Fedora Project.
"As long as people keep putting out new versions and new browsers, like now we have the new Chrome browser, there is always catching up to do. Selenium will have to make sure that it keeps working," Huggins said.
The community version still uses the five-year-old code line of version 1.0. However, Sauce Labs, Selenium community members and a few engineers at Google are readying version 2.0.
With the release of the new code base in Selenium 2.0, Sauce Labs is merging the Selenium Project with the WebDriver project, sponsored by Google. This will bring a better engine under the hood of Selenium, according to Huggins.
A good analogy to the benefits of the software merger is when Ruby on Rails merged with Merv. Merv was a better and faster engine under the hood, but Rails had better recognition as the brand, Higgins noted. <
WebDriver automates testing Web applications and verifies that they work as expected. It's not tied to any particular test framework. This makes it suitable for use with other test beds.
Merging WebDriver with Selenium 2.0 demonstrates a rare example of open source projects collaborating rather than competing with one another. More importantly, it is an example of being able to take the best of both worlds, Higgins said.
"The result is that Selenium 2.0 is going to be a much more robust and feature-rich offering to users that will provide compelling advantages compared to some of the proprietary offerings in this space," he added.
A Marriage Made
Selenium as it is solves about 80 percent of the existing cross-platform problems that software developers face. WebDriver adds Web browser testing in a different way in terms of its underlying technology.
It calls more to the operating system or lower system level. Selenium actually implements calls on a higher level. Because of this, Selenium has more limitations in what it can do, explained Huggins.
For example, with Selenium if a test generated a warning dialog about a Web site's certificate, Selenium would not have a dialog box to respond. WebDriver does. It's kind of like the technology to solve that last mile, the last 20 percent, he added.
On Feb. 12 the company released Sauce RC (Remote Control) 1.0. This is a commercially supported free Selenium distribution that installs easily and runs on all the latest browser releases. Access to expert support directly from the people who built Selenium is also available.
Selenium RC is a developer-focused tool that is part of the Selenium functional testing framework for Web applications. Configuration of Sauce RC is done via a Web interface that allows for remote administration of test servers.