Welcome!

Open Source Cloud Authors: Elizabeth White, Pat Romanski, Greg Schulz, Carmen Gonzalez, Patrick Hubbard

Related Topics: FinTech Journal, Open Source Cloud, @CloudExpo

FinTech Journal: Blog Post

Tap-as-a-Service: Enabling Traffic Monitoring in OpenStack Clouds | @CloudExpo #API #Cloud

Tap-as-a-Service, a platform-oriented solution, designed to operate as an extension of Neutron, the OpenStack network service

It is no surprise that OpenStack has evolved into a widely adopted cloud management framework. As it hurtles on a trajectory of rapid growth, a new breed of demands are making themselves felt - demands that a mature platform of this nature and scope must satisfy. One such requirement is the ability to monitor traffic flowing in the myriad of virtual networks found in an OpenStack datacenter.

Conceptually speaking, the monitoring process involves placing tap devices at appropriate locations within the network infrastructure and attaching traffic analyzers to them. These analyzers can then see the same packets passing through those network segments, as if they were also in-line. A logical tap device can be easily constructed using the port-mirroring function of a network switching element, which makes it possible to have a copy of the packets traversing one or more switch ports delivered to another port on the same switch. This capability is supported by almost all modern physical and virtual switches. So, why is it [still] not possible to monitor the activity in OpenStack virtual networks?

The answer to this question lies in understanding two important architectural characteristics of cloud-based virtualization platforms, namely multi-tenancy and location independence. The former allows available resources and services to be shared among different groups of users. Each group - known as a tenant - is provided with an environment that is completely isolated from the others, to the extent where members of a group are oblivious of the fact that other groups may be co-existing with them. Multi-tenancy promotes delegation of control in a safe and secure manner. For example, tenants are permitted to create and administer their own private virtual networks. Location independence, on the other hand, is primarily concerned with hiding the identities of individual infrastructure components from the virtualized workloads. This has made it possible to relocate running virtual machines from one host to another. An equally important but perhaps less appreciated benefit of location independence is improved efficiency in resource allocation. Tenants are therefore unaware of the physical hosts on which their virtual machines are running. Furthermore, virtual machines belonging to different tenants may be placed on the same host. In such a shared ecosystem it makes sense that tenants are not given direct access to the underlying switch fabric, consisting of host-level virtual switches, top-of-rack switches, etc. This restriction avoids the possibility of any cross-tenant data leakage. Unfortunately, this means that the port-mirroring capability of those switches is also not available.

OpenStack is not alone when it comes to the lack of traffic monitoring support for its virtual networks. Other cloud solutions, including Amazon Web Services (AWS), also suffer from this limitation for the very reasons described above. There is one aspect of OpenStack, however, that makes it stand out from the rest. It is open source technology! This provides an opportunity for someone (actually anyone) who is able and willing to introduce new capabilities into the platform. At Gigamon we live and breathe network traffic visibility, so we decided to take it upon ourselves and become ‘that someone' who will push the needle forward. To our surprise and delight we soon learned of a group at Ericsson who had independently arrived at the same conclusion. Our goals matched perfectly and it seemed natural and fitting that we pool our resources together to solve this problem.

Our project is called Tap-as-a-Service. It is a platform-oriented solution, designed to operate as an extension of Neutron, the OpenStack network service. TaaS offers a simple API that will enable a tenant (or the cloud administrator) to monitor ports in Neutron provisioned networks. Since it is vital that tenant boundaries are not compromised, a tenant can only monitor its own ports, i.e., any port on one of its private virtual networks or a port created by it on a shared virtual network. The TaaS workflow begins with the creation of a tap-service instance that has a Neutron port serving as the destination side of a port-mirror session. A monitoring virtual machine is usually attached to this port to consume the mirrored traffic. Later, one or more tap-flows can be added to the tap-service instance. A tap-flow represents the association between a (source) port that is being monitored and a tap-service instance. TaaS allows a mirror session to span across multiple hosts, by virtue of remote port-mirroring, thereby ensuring that location independence is preserved.

A reference implementation of TaaS was completed earlier this year and the source code has been uploaded to Stackforge, the OpenStack incubator, where a dedicated GIT repository now exists for this project [1]. At the last OpenStack Summit in Vancouver (May 2015), we did a technical presentation on this work that included a live demonstration of traffic monitoring using TaaS [2]. The response has been very positive, with support pouring in from both the developer and user communities. We will continue to enhance TaaS; some of the planned features are integration with the OpenStack dashboard, support for virtual machine migration, pre-capture filtering and rate-limiting of mirrored traffic. At the same time, we are also continuing our discussions with the Neutron core team to have TaaS accepted as an integral part of future OpenStack releases.

Port-mirroring used to be a switch layer function. Tap-as-a-Service has effectively virtualized this capability and made it available for the users of Neutron provisioned networks. We see TaaS as the basic building block on top of which more complex traffic visibility solutions can be engineered for a diverse set of use cases, ranging from network administration and trouble-shooting to application/network security, data analytics and more.

References

  1. Tap-as-a-Service code repository.
  2. Tap-as-a-Service (TaaS): Port Monitoring for Neutron Networks. Alan Kavanagh, Anil Rao, Vinay Yadhav, OpenStack Summit, Vancouver, Canada, May 20, 2015.

More Stories By Anil Rao

Anil Rao is a Distinguished Engineer at Gigamon, leading research and development activities related to virtual machine traffic monitoring. His research interests include operating systems, distributed computing, virtual machine technology and software defined networking.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloudistics delivers a complete public cloud experience with composable on-premises infrastructures to medium and large enterprises. Its software-defined technology natively converges network, storage, compute, virtualization, and management into a ...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs oft...
As cloud adoption continues to transform business, today's global enterprises are challenged with managing a growing amount of information living outside of the data center. The rapid adoption of IoT and increasingly mobile workforce are exacerbating the problem. Ensuring secure data sharing and efficient backup poses capacity and bandwidth considerations as well as policy and regulatory compliance issues.
SYS-CON Events announced today that A&I Solutions has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 1999, A&I Solutions is a leading information technology (IT) software and services provider focusing on best-in-class enterprise solutions. By partnering with industry leaders in technology, A&I assures customers high performance levels across all IT environments including: ma...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT serv...
SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
SYS-CON Events announced today that Interoute has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Interoute is the owner operator of Europe's largest network and a global cloud services platform, which encompasses over 70,000 km of lit fiber, 15 data centers, 17 virtual data centers and 33 colocation centers, with connections to 195 additional partner data centers. Our full-service Unifie...
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo Silicon Valley Call for Papers is now open.
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive ad...
Everywhere we turn in our industry we can find strong opinions about the direction, type and nature of cloud’s impact on computing and business. Another word that is used in every context in our industry is “hybrid.” In his session at 20th Cloud Expo, Alvaro Gonzalez, Director of Technical, Partner and Field Marketing at Peak 10, will use a combination of a few conceptual props and some research recently commissioned by Peak 10 to offer a real-world consideration of how the various categories of...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deli...