Welcome!

Open Source Cloud Authors: Pat Romanski, Elizabeth White, Ed Featherston, Roger Strukhoff, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Web Browser Automation Tools

There are many Web Browser Automation Tools, both paid and free / open source

Web Browser Automation is a process where certain steps in the web browser are performed repetitively to ensure the correct operation of the web application’s functionality. It may be applied for QA testing in the development process and for control over Information System accessibility and performance during implementation. The second becomes more and more important since current market trends indicate that just having a good service is not enough; the service should also be highly accessible and effective. With Web Browser Automation Tools it is possible to check accessibility and performance by periodically running some transaction scenarios for certain services.

There are many Web Browser Automation Tools, both paid and free / open source. The main purpose of these tools is to record a specific transaction scenario for a browser, play it back by injecting JavaScript into web pages and then to provide the corresponding results. The tools vary in many characteristics: OS platform support, supported programming languages, SSL support, whether it uses a real web browser or simulates the real browser’s behavior, etc. Below are presented a relative comparison of some popular free/open source tool’s characteristics.

Tool Simulation Supported Browsers Supported Languages Test Recorder
Selenium No Firefox, IE(partially), Opera (partially), Chrome (partially)[1] C#, Java, Perl, PHP, Python, Ruby Yes
Watij No On Windows only IE, On Linux only Firefox,
On Mac only Safari
java No
Windmill No Python and JavaScript Python, JavaScript, Ruby Yes
Watir No On Windows only IE, Watir Web Driver -IE/FF/Chrome/Safari Ruby Yes
Watin No IE, Chrome, Firefox .net platform languages Yes
Sahi No IE, Safari, Firefox, Chrome, Opera Java, Javascript Yes
Canoo WebTest Yes No need Javascript, Python Yes
CasperJS Yes No need Javascript Yes

Selenium’s different products are compatible, at least partially, with a variety of different browsers, but all of them are fully compatible with Firefox.


Aside from their variation in functionality, these tools have another essential difference. Some of them are headless, which means that they simulate browser behavior. For example, the CasperJS tool uses WebKit engine capabilities to simulate certain test cases. On one hand, headless tools are more productive, but on the other hand, the simulation may generate different results than those of the real browser.
Different tools have their specific advantages. Some of them work with different browsers without significant deviation or trouble; others provide higher performance, etc. There are many articles/posts comparing the pros and cons of web automation tools, but it would also be interesting to compare their performance. We have tested 3 different tools: Selenium, Watij and Casperjs on the same host with the following parameters: Windows 7 32 bit, 4 Gb RAM and the Intel i5 CPU. From these 3 tools, Selenium may currently be the most popular and richly featured, and it offers several products such as IDE, API, Remote Control server (to accept HTTP requests for the browser), Grid server (to test different browser instances running on remote machines), etc. Casperjs is a headless Phantomjs based tool and the Watij is Watir’s java version based on JXbrowser (a rich web browser component to integrate to Java).

To test the tools, we used 45 transaction instances with the same scenario of 7 steps (navigation, clicks, login, logout, etc.). A single transaction execution period was 5 minutes and the overall test duration was 20 minutes. Each tool was tested 2 times: first by running with 1 thread and then with 5 threads. The test results are shown below:
Tool Num. of tests Browsers App run duration Num. of threads Test frequency (min) Transaction step count Num. of tests running during 20 mins Avg. exec time for 1 test (sec)
Watij 45 IE 20 min 5 5 7 34 29.7
Watij 45 IE 20 min 1 5 7 32 33.2
Selenium 45 IE 20 min 5 5 7 - -
Selenium 45 IE 20 min 1 5 7 68 17.6
CasperJS 45 NA 20 min 5 5 7 204 17.1
CasperJS 45 NA 20 min 1 5 7 68 17.2

As you can see, the test results for Selenium with IE browser are absent. The problem was that the Selenium couldn’t run several transaction tests at the same time. Instead of opening several browser instances, it mixes all of them in one browser instance. However, it doesn’t have this kind of problem when working with Firefox or Chrome. For example, running the Selenium with the same conditions but on the Firefox browser, the following results were obtained: average execution time of 1 test is 15.1 seconds, the number of tests run during 20 minutes is 50. The average running time of one transaction test for these tools is not greater than that of the others. The other 2 tools spent a substantial amount of overhead time communicating with the server, starting browser instances, etc.

Examining the performance of selected Web Automation Tools shows that they differ not only in terms of functionality, but that they also behave differently in different conditions (depending on OS, browser type, multithread or singlethread). The main limitation we noticed with Selenium is that it’s primarily designed for Firefox. All of its products support Firefox, and as we have already seen it has problems working with IE. On the other hand the Casperjs is headless and is Webkit based, so it will not work on IE only sites, for example. Meanwhile, Watij works with only one unique browser type per OS (IE on Windows, Firefox on Linux and Safari on Mac). Working with different transaction scenarios simultaneously is much more productive with the headless tools since they consume less time.

 

Read the original blog entry...

More Stories By Hovhannes Avoyan

Hovhannes Avoyan is the CEO of PicsArt, Inc.,

@ThingsExpo Stories
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effi...
Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2016 Silicon Valley. The 6thInternet of @ThingsExpo will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Presidio has received the 2015 EMC Partner Services Quality Award from EMC Corporation for achieving outstanding service excellence and customer satisfaction as measured by the EMC Partner Services Quality (PSQ) program. Presidio was also honored as the 2015 EMC Americas Marketing Excellence Partner of the Year and 2015 Mid-Market East Partner of the Year. The EMC PSQ program is a project-specific survey program designed for partners with Service Partner designations to solicit customer feedbac...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, provided an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profession...
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
Connected devices and the industrial internet are growing exponentially every year with Cisco expecting 50 billion devices to be in operation by 2020. In this period of growth, location-based insights are becoming invaluable to many businesses as they adopt new connected technologies. Knowing when and where these devices connect from is critical for a number of scenarios in supply chain management, disaster management, emergency response, M2M, location marketing and more. In his session at @Th...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
There are several IoTs: the Industrial Internet, Consumer Wearables, Wearables and Healthcare, Supply Chains, and the movement toward Smart Grids, Cities, Regions, and Nations. There are competing communications standards every step of the way, a bewildering array of sensors and devices, and an entire world of competing data analytics platforms. To some this appears to be chaos. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Bradley Holt, Developer Advocate a...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Apixio Inc. has raised $19.3 million in Series D venture capital funding led by SSM Partners with participation from First Analysis, Bain Capital Ventures and Apixio’s largest angel investor. Apixio will dedicate the proceeds toward advancing and scaling products powered by its cognitive computing platform, further enabling insights for optimal patient care. The Series D funding comes as Apixio experiences strong momentum and increasing demand for its HCC Profiler solution, which mines unstruc...
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2016 Silicon Valley. The 19th Cloud Expo and 6th @ThingsExpo will take place on November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. "The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Interne...
In addition to all the benefits, IoT is also bringing new kind of customer experience challenges - cars that unlock themselves, thermostats turning houses into saunas and baby video monitors broadcasting over the internet. This list can only increase because while IoT services should be intuitive and simple to use, the delivery ecosystem is a myriad of potential problems as IoT explodes complexity. So finding a performance issue is like finding the proverbial needle in the haystack.
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to imp...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
IoT offers a value of almost $4 trillion to the manufacturing industry through platforms that can improve margins, optimize operations & drive high performance work teams. By using IoT technologies as a foundation, manufacturing customers are integrating worker safety with manufacturing systems, driving deep collaboration and utilizing analytics to exponentially increased per-unit margins. However, as Benoit Lheureux, the VP for Research at Gartner points out, “IoT project implementers often ...
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...