Welcome!

Open Source Cloud Authors: Pat Romanski, Yeshim Deniz, Liz McMillan, Stackify Blog, Wesley Coelho

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Blog Feed Post

The Speed Index | @DevOpsSummit @Catchpoint #DevOps #WebPerf #DX #AI

There’s a lot of buzz around the speed index metric, but what exactly is this metric and how is it calculated?

Understanding Speed Index
By Moiz Khan

There’s a lot of buzz around the speed index metric, but what exactly is this metric and how is it calculated? Even more so, why do we need a new metric when there are so many already on the plate? This article gives you detailed answers to these questions.

What is speed index?
User expectations have changed drastically, webpages are expected to load faster with all elements rendering as soon as possible; user frustration increases if the visual content of the page is slow to load.

There are several metrics used to measure user experience, but do these give us a clear enough picture? Not really. For example, render start indicates when the page started painting, but this could mean it loaded a white background, a random object, or just a dot on the page; the user, however, may still be looking at a blank page. The document complete metric, which flags the onLoad event, may not be important from an end user’s perspective as it considers below-the-fold requests as well; there could be instances where the Document complete is triggered even when the page is blank.

Speed Index is an abstract score that was introduced to overcome the limitations of the current set of metrics. It measures user experience and unlike render start or document complete, it is not a timing metric, therefore the smaller the score, the better it is. Speed index takes into account the visual completeness of the page above-the-fold. It does a much better job of measuring the perceived performance of a page, but does this mean other metrics are irrelevant? This is debatable. Instead, the choice of a metric depends on the scenario and the nature of the website being analyzed.

How relevant is speed index?
Speed index will let you know if the visible content is loading quickly or not. The lower the score, the better is the user experience, and vice versa. The speed index score indicates whether you need to optimize your page or not. To maintain a good score, there are two critical aspects that you should look at: optimizing content efficiency and optimizing the critical rendering path.

Web applications continue to grow in terms of functionality, scope, and usability; hundreds of resources make up these applications and fetching each resource adds to the page load time. Maintaining performance is possible only when the resources and content that make up the page are optimized. Speed index plays a critical role in monitoring the performance of visible content.

How is it calculated?
Speed index lets you measure how quickly a page’s content is visible. However, it can be a little tricky to understand what it means and what factors contribute to the score.

Speed index uses film strips to calculate the score, each frame is scored for visual completeness above the fold – the score is 0% for a blank screen and 100% for a visually complete screen. The score for each frame is calculated using this formula:

interval time*(1 – visual complete %/100)

The process is repeated for every frame and the cumulative of all the scores gives you the speed index of that page.

Let’s take an example:

Adding all of the scores gives the speed index score:

500 + 450 + 350 + 200 + 50 = 1550

Here is another example:

Let’s add up the score:

500 + 500 + 475 + 400 + 350 = 2225

From a performance point of view, the lower the score the better it is. There is no defined benchmark for speed index, however, a score of <1000 is considered good. User experience is better if above-the-fold content is displayed faster, and this is what speed index determines.

Measuring speed index using Catchpoint
To check speed index, you will need to enable the “Capture Filmstrip” option under Advanced Settings of a test. It is displayed along with other metrics in the charts, and you can also view it in the waterfall graphs. Here are a few examples that illustrate how pages that load the visual content faster have a better speed index score.

Example 1:

In the above example, the page has a good speed index score of 1260. If you look at the film strip, the interval is of 500ms, and from the second frame the visual content is visible, which is about 1 second. Frame three has almost all the content loaded. From a user’s point of view, we have engaged them from the very first second due to the fact that the page is optimized.

Example 2:

In the above image, the visual is visible in the third frame. The first two frames are blank, which is why the speed index score of 1741 is higher than the first example. The user views content at 1.6 seconds, unlike the first example where the user viewed the content within a second.

Example 3:

Based on the final example above, we are able to understand what is happening. The first two frames are blank and in the third frame visual content starts to load. However, the completeness is less compared to the other two images in the same frame. The visual content continues to load in the fourth frame, but it’s not complete until the fifth frame. This means that the user needs to wait for 1.6 seconds to see something on the page, and another 2.8 seconds to view the complete visual content.

In all of the examples, the render start is triggered before the user views anything on the page. In the second example, the render start happens at 1195ms and the visible content starts to load at 1604ms. In the third example, the render start happens at 1296ms and the visible content starts to load at 1597ms. This is the uniqueness of speed index. With a highly unoptimized website, the difference could be extremely high, with render start happening quickly but the visible content loads quite late. You need these insights to optimize your websites.

Limitations of Speed Index
When running transaction tests for dynamic sites or single page applications that use ajax, the score is always good in the steps that follow once the page is opened. This is because the page does not refresh after it is loaded. Pages featuring carousels that rotate automatically may be penalized because they continue to change even after the page load is completed.

Summary
Speed index indicates the visual completeness of the page; however, it does not indicate if the content was critical or non-critical for the user. As such, it should not be a replacement for other metrics and should be considered a rather useful and much-needed addition to the list of metrics. You will still need to look at render start and document complete to understand the overall performance. Speed index gives you a different perspective of the user experience that your site delivers.

The post Understanding Speed Index appeared first on Catchpoint's Blog - Web Performance Monitoring.

More Stories By Mehdi Daoudi

Catchpoint radically transforms the way businesses manage, monitor, and test the performance of online applications. Truly understand and improve user experience with clear visibility into complex, distributed online systems.

Founded in 2008 by four DoubleClick / Google executives with a passion for speed, reliability and overall better online experiences, Catchpoint has now become the most innovative provider of web performance testing and monitoring solutions. We are a team with expertise in designing, building, operating, scaling and monitoring highly transactional Internet services used by thousands of companies and impacting the experience of millions of users. Catchpoint is funded by top-tier venture capital firm, Battery Ventures, which has invested in category leaders such as Akamai, Omniture (Adobe Systems), Optimizely, Tealium, BazaarVoice, Marketo and many more.

@ThingsExpo Stories
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
As popularity of the smart home is growing and continues to go mainstream, technological factors play a greater role. The IoT protocol houses the interoperability battery consumption, security, and configuration of a smart home device, and it can be difficult for companies to choose the right kind for their product. For both DIY and professionally installed smart homes, developers need to consider each of these elements for their product to be successful in the market and current smart homes.
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant th...
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its ...
SYS-CON Events announced today that Golden Gate University will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Since 1901, non-profit Golden Gate University (GGU) has been helping adults achieve their professional goals by providing high quality, practice-based undergraduate and graduate educational programs in law, taxation, business and related professions. Many of its courses are taug...
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, will discuss how by using...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
SYS-CON Events announced today that CAST Software will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CAST was founded more than 25 years ago to make the invisible visible. Built around the idea that even the best analytics on the market still leave blind spots for technical teams looking to deliver better software and prevent outages, CAST provides the software intelligence that matter ...
SYS-CON Events announced today that Daiya Industry will exhibit at the Japanese Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ruby Development Inc. builds new services in short period of time and provides a continuous support of those services based on Ruby on Rails. For more information, please visit https://github.com/RubyDevInc.