Welcome!

Open Source Cloud Authors: Pat Romanski, Yeshim Deniz, Liz McMillan, Stackify Blog, Wesley Coelho

Related Topics: FinTech Journal, Open Source Cloud, @CloudExpo

FinTech Journal: Blog Post

Tap-as-a-Service: Enabling Traffic Monitoring in OpenStack Clouds | @CloudExpo #API #Cloud

Tap-as-a-Service, a platform-oriented solution, designed to operate as an extension of Neutron, the OpenStack network service

It is no surprise that OpenStack has evolved into a widely adopted cloud management framework. As it hurtles on a trajectory of rapid growth, a new breed of demands are making themselves felt - demands that a mature platform of this nature and scope must satisfy. One such requirement is the ability to monitor traffic flowing in the myriad of virtual networks found in an OpenStack datacenter.

Conceptually speaking, the monitoring process involves placing tap devices at appropriate locations within the network infrastructure and attaching traffic analyzers to them. These analyzers can then see the same packets passing through those network segments, as if they were also in-line. A logical tap device can be easily constructed using the port-mirroring function of a network switching element, which makes it possible to have a copy of the packets traversing one or more switch ports delivered to another port on the same switch. This capability is supported by almost all modern physical and virtual switches. So, why is it [still] not possible to monitor the activity in OpenStack virtual networks?

The answer to this question lies in understanding two important architectural characteristics of cloud-based virtualization platforms, namely multi-tenancy and location independence. The former allows available resources and services to be shared among different groups of users. Each group - known as a tenant - is provided with an environment that is completely isolated from the others, to the extent where members of a group are oblivious of the fact that other groups may be co-existing with them. Multi-tenancy promotes delegation of control in a safe and secure manner. For example, tenants are permitted to create and administer their own private virtual networks. Location independence, on the other hand, is primarily concerned with hiding the identities of individual infrastructure components from the virtualized workloads. This has made it possible to relocate running virtual machines from one host to another. An equally important but perhaps less appreciated benefit of location independence is improved efficiency in resource allocation. Tenants are therefore unaware of the physical hosts on which their virtual machines are running. Furthermore, virtual machines belonging to different tenants may be placed on the same host. In such a shared ecosystem it makes sense that tenants are not given direct access to the underlying switch fabric, consisting of host-level virtual switches, top-of-rack switches, etc. This restriction avoids the possibility of any cross-tenant data leakage. Unfortunately, this means that the port-mirroring capability of those switches is also not available.

OpenStack is not alone when it comes to the lack of traffic monitoring support for its virtual networks. Other cloud solutions, including Amazon Web Services (AWS), also suffer from this limitation for the very reasons described above. There is one aspect of OpenStack, however, that makes it stand out from the rest. It is open source technology! This provides an opportunity for someone (actually anyone) who is able and willing to introduce new capabilities into the platform. At Gigamon we live and breathe network traffic visibility, so we decided to take it upon ourselves and become ‘that someone' who will push the needle forward. To our surprise and delight we soon learned of a group at Ericsson who had independently arrived at the same conclusion. Our goals matched perfectly and it seemed natural and fitting that we pool our resources together to solve this problem.

Our project is called Tap-as-a-Service. It is a platform-oriented solution, designed to operate as an extension of Neutron, the OpenStack network service. TaaS offers a simple API that will enable a tenant (or the cloud administrator) to monitor ports in Neutron provisioned networks. Since it is vital that tenant boundaries are not compromised, a tenant can only monitor its own ports, i.e., any port on one of its private virtual networks or a port created by it on a shared virtual network. The TaaS workflow begins with the creation of a tap-service instance that has a Neutron port serving as the destination side of a port-mirror session. A monitoring virtual machine is usually attached to this port to consume the mirrored traffic. Later, one or more tap-flows can be added to the tap-service instance. A tap-flow represents the association between a (source) port that is being monitored and a tap-service instance. TaaS allows a mirror session to span across multiple hosts, by virtue of remote port-mirroring, thereby ensuring that location independence is preserved.

A reference implementation of TaaS was completed earlier this year and the source code has been uploaded to Stackforge, the OpenStack incubator, where a dedicated GIT repository now exists for this project [1]. At the last OpenStack Summit in Vancouver (May 2015), we did a technical presentation on this work that included a live demonstration of traffic monitoring using TaaS [2]. The response has been very positive, with support pouring in from both the developer and user communities. We will continue to enhance TaaS; some of the planned features are integration with the OpenStack dashboard, support for virtual machine migration, pre-capture filtering and rate-limiting of mirrored traffic. At the same time, we are also continuing our discussions with the Neutron core team to have TaaS accepted as an integral part of future OpenStack releases.

Port-mirroring used to be a switch layer function. Tap-as-a-Service has effectively virtualized this capability and made it available for the users of Neutron provisioned networks. We see TaaS as the basic building block on top of which more complex traffic visibility solutions can be engineered for a diverse set of use cases, ranging from network administration and trouble-shooting to application/network security, data analytics and more.

References

  1. Tap-as-a-Service code repository.
  2. Tap-as-a-Service (TaaS): Port Monitoring for Neutron Networks. Alan Kavanagh, Anil Rao, Vinay Yadhav, OpenStack Summit, Vancouver, Canada, May 20, 2015.

More Stories By Anil Rao

Anil Rao is a Distinguished Engineer at Gigamon, leading research and development activities related to virtual machine traffic monitoring. His research interests include operating systems, distributed computing, virtual machine technology and software defined networking.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they b...
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TAZMO technology and development capabilities in the semiconductor and LCD-related manufacturing fields are among the best worldwide. For more information, visit https://www.tazmo.co.jp/en/.
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its ...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http:...
Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, will discuss how by using...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
SYS-CON Events announced today that Daiya Industry will exhibit at the Japanese Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ruby Development Inc. builds new services in short period of time and provides a continuous support of those services based on Ruby on Rails. For more information, please visit https://github.com/RubyDevInc.
As businesses evolve, they need technology that is simple to help them succeed today and flexible enough to help them build for tomorrow. Chrome is fit for the workplace of the future — providing a secure, consistent user experience across a range of devices that can be used anywhere. In her session at 21st Cloud Expo, Vidya Nagarajan, a Senior Product Manager at Google, will take a look at various options as to how ChromeOS can be leveraged to interact with people on the devices, and formats th...