Welcome!

Open Source Cloud Authors: Pat Romanski, Liz McMillan, Yeshim Deniz, Zakia Bouachraoui, Elizabeth White

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Web Browser Automation Tools

There are many Web Browser Automation Tools, both paid and free / open source

Web Browser Automation is a process where certain steps in the web browser are performed repetitively to ensure the correct operation of the web application’s functionality. It may be applied for QA testing in the development process and for control over Information System accessibility and performance during implementation. The second becomes more and more important since current market trends indicate that just having a good service is not enough; the service should also be highly accessible and effective. With Web Browser Automation Tools it is possible to check accessibility and performance by periodically running some transaction scenarios for certain services.

There are many Web Browser Automation Tools, both paid and free / open source. The main purpose of these tools is to record a specific transaction scenario for a browser, play it back by injecting JavaScript into web pages and then to provide the corresponding results. The tools vary in many characteristics: OS platform support, supported programming languages, SSL support, whether it uses a real web browser or simulates the real browser’s behavior, etc. Below are presented a relative comparison of some popular free/open source tool’s characteristics.

Tool Simulation Supported Browsers Supported Languages Test Recorder
Selenium No Firefox, IE(partially), Opera (partially), Chrome (partially)[1] C#, Java, Perl, PHP, Python, Ruby Yes
Watij No On Windows only IE, On Linux only Firefox,
On Mac only Safari
java No
Windmill No Python and JavaScript Python, JavaScript, Ruby Yes
Watir No On Windows only IE, Watir Web Driver -IE/FF/Chrome/Safari Ruby Yes
Watin No IE, Chrome, Firefox .net platform languages Yes
Sahi No IE, Safari, Firefox, Chrome, Opera Java, Javascript Yes
Canoo WebTest Yes No need Javascript, Python Yes
CasperJS Yes No need Javascript Yes

Selenium’s different products are compatible, at least partially, with a variety of different browsers, but all of them are fully compatible with Firefox.


Aside from their variation in functionality, these tools have another essential difference. Some of them are headless, which means that they simulate browser behavior. For example, the CasperJS tool uses WebKit engine capabilities to simulate certain test cases. On one hand, headless tools are more productive, but on the other hand, the simulation may generate different results than those of the real browser.
Different tools have their specific advantages. Some of them work with different browsers without significant deviation or trouble; others provide higher performance, etc. There are many articles/posts comparing the pros and cons of web automation tools, but it would also be interesting to compare their performance. We have tested 3 different tools: Selenium, Watij and Casperjs on the same host with the following parameters: Windows 7 32 bit, 4 Gb RAM and the Intel i5 CPU. From these 3 tools, Selenium may currently be the most popular and richly featured, and it offers several products such as IDE, API, Remote Control server (to accept HTTP requests for the browser), Grid server (to test different browser instances running on remote machines), etc. Casperjs is a headless Phantomjs based tool and the Watij is Watir’s java version based on JXbrowser (a rich web browser component to integrate to Java).

To test the tools, we used 45 transaction instances with the same scenario of 7 steps (navigation, clicks, login, logout, etc.). A single transaction execution period was 5 minutes and the overall test duration was 20 minutes. Each tool was tested 2 times: first by running with 1 thread and then with 5 threads. The test results are shown below:
Tool Num. of tests Browsers App run duration Num. of threads Test frequency (min) Transaction step count Num. of tests running during 20 mins Avg. exec time for 1 test (sec)
Watij 45 IE 20 min 5 5 7 34 29.7
Watij 45 IE 20 min 1 5 7 32 33.2
Selenium 45 IE 20 min 5 5 7 - -
Selenium 45 IE 20 min 1 5 7 68 17.6
CasperJS 45 NA 20 min 5 5 7 204 17.1
CasperJS 45 NA 20 min 1 5 7 68 17.2

As you can see, the test results for Selenium with IE browser are absent. The problem was that the Selenium couldn’t run several transaction tests at the same time. Instead of opening several browser instances, it mixes all of them in one browser instance. However, it doesn’t have this kind of problem when working with Firefox or Chrome. For example, running the Selenium with the same conditions but on the Firefox browser, the following results were obtained: average execution time of 1 test is 15.1 seconds, the number of tests run during 20 minutes is 50. The average running time of one transaction test for these tools is not greater than that of the others. The other 2 tools spent a substantial amount of overhead time communicating with the server, starting browser instances, etc.

Examining the performance of selected Web Automation Tools shows that they differ not only in terms of functionality, but that they also behave differently in different conditions (depending on OS, browser type, multithread or singlethread). The main limitation we noticed with Selenium is that it’s primarily designed for Firefox. All of its products support Firefox, and as we have already seen it has problems working with IE. On the other hand the Casperjs is headless and is Webkit based, so it will not work on IE only sites, for example. Meanwhile, Watij works with only one unique browser type per OS (IE on Windows, Firefox on Linux and Safari on Mac). Working with different transaction scenarios simultaneously is much more productive with the headless tools since they consume less time.

 

Read the original blog entry...

More Stories By Hovhannes Avoyan

Hovhannes Avoyan is the CEO of PicsArt, Inc.,

IoT & Smart Cities Stories
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
DXWorldEXPO LLC announced today that Telecom Reseller has been named "Media Sponsor" of CloudEXPO | DXWorldEXPO 2018 New York, which will take place on November 11-13, 2018 in New York City, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to great conferences, helping you discover new conferences and increase your return on investment.
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.