Click here to close now.

Welcome!

Open Source Authors: Roger Strukhoff, Liz McMillan, Elizabeth White, Carmen Gonzalez, Pat Romanski

Blog Feed Post

How I built a monitor for RabbitMQ

When you are going to monitor your engine, you probably try first to find some tool that can do it. Very likely, of course, that tool should be able to handle monitoring online and to notify you when the health of your engine is starting to go bad. There are already several well-known and widespread online monitoring tools available for many systems. Super! But what if I try to find a suitable tool for my specific application and… ohh, failure – I cannot find what I need. Unfortunately, it’s also very likely that no one can even suggest a suitable tool. What can I do in such a situation? Probably try to do it myself. Who can help me? You’ve probably already guessed that I am talking about Monitis.  You may ask “why Monitis?”  Because Monitis suggests Monitis Open API, and that gives you a chance to build any monitoring tool yourself. In line with Monitis slang, such a monitor has been named the custom monitor.

But let’s cease the generic talk and get back to my story.

Recently, I had to use RabbitMQ server in one of my projects, and naturally I had a wish to monitor it; just to measure data that I want to monitor. RabbitMQ contains a nice plugin named RabbitMQ Management that in fact provides monitoring of RabbitMQ via any browser (thanks to the embedded WEBUI). Well, it is undoubtedly good, but I don’t want to sit whole days next to the monitor waiting for a problem to come in. I want to monitor online, to have the possibility to go back in time by viewing the monitoring history and, why not, to get a notification on my mobile while having a beer in the bar. I think every admin dreams of such a life.

Well, since I use an Ubuntu server I have decided to implement my custom monitor as Bash script to avoid any unnecessary dependencies and to take into account that the Bash script wrapper for Monitis API  is already implemented.

First of all I have to install RabbitMQ server. The easiest way to do this is to download the “deb” package from the original RabbitMQ site and install it by using the following command:

 

sudo dpkg -i rabbitmq-server_2.8.x_all.deb

 

So far so good. Next step – enabling the RabbitMQ Management HTTP API  that allows, in addition, getting necessary information by using REST technology. The management plugin has been included in the RabbitMQ distribution since version 2.8.1. To enable it, use the following command:

 

sudo rabbitmq-plugins enable rabbitmq_management

 

Please note that for older versions you have to install this plugin separately.

That’s all. Now I can use the RabbitMQ server by writing the following command to control RabbitMQ:

 

sudo /etc/init.d/rabbitmq-server {start|stop|status|rotate-logs|restart}

 

For instance, RabbitMQ server status will be shown as depicted below:

 

 

All is okay up to now. Well, after some investigation I have decided to measure the following available metrics:

  •  osd_pr – The percentage of open socket descriptors RabbitMQ server to the allowed maximum number of open sockets by process.
  •  ofd_pr – The percentage of open file descriptors RabbitMQ server to the allowed maximum number of open files by process.
  •  cpu_usage – the percentage of cpu usage by RabbitMQ server.
  •  mem_usage – the percentage of memory usage by RabbitMQ server.
  •  msg_in_queue – the number of messages that are still in the queue.
  •  timeout – queues timeout in seconds.
  •  pub_rate – Average value of total published messages into queues per second.
  •  from_client_rate – Total inbound throughput value estimated in Kbytes per second.
  •  to_client_rate – Total outbound throughput value estimated in Kbytes per second.
  •  get_rate – Average value of total got messages from queues per second.
  •  status – the evaluation of health status of RabbitMQ server (OK, IDLE, NOK, FAIL)

 

The health status of RabbitMQ should be evaluated as ‘NOK’ when at least one of the following events is detected:

  • The percentage of open file descriptors (ofd_pr) exceeds 90%
  • The percentage of open socket descriptors (osd_pr) exceeds 90%
  • The percentage of used Erlang processes to available Erlang processes exceeds 90%
  • The percentage of memory usage (mem_usage) exceeds 95%
  • The percentage of cpu usage (cpu_usage) exceeds 95%
  • There are messages in the queue (msg_in_queue > 0)

The health status ‘FAIL’ should be generated when RabbitMQ server is unavailable for some reason and the health status ‘IDLE’ should be generated when RabbitMQ server isn’t receiving any messages from clients.

It seems that is all I want for now. Okay, I developed quite quickly such a Bash Script for monitoring and got the following set of files:

monitis_api.sh         Monitis API wrapper implementation

monitis_util.sh        Utilities function set

monitis_global.sh      Monitis API wrapper global variables

monitis_constant.sh    Monitis API constants

rabbitmq_monitor.sh    RabbitMQ custom monitor implementation

monitor_constant.sh    RabbitMQ monitor constants

rmqmon_start.sh        Main executable script

ticktick.sh            Bash JSON parser

Note that I have really developed only the “rabbitmq_monitor.sh” script. The other scripts were simply adapted. Please also notice that I have to use the third party open source JSON parser (to allow processing JSON in bash script) because RabbitMQ Management HTTP API responses are in the JSON form.

 

Well, now it’s time for testing.

I have prepared two clients on JavaScript (Node.js) and Java for connecting to RabbitMQ. In addition, the load simulator was prepared in a way that is intended to generate quite a big load for processing.

So, I have run the monitor and simulator and after some time opened my dashboard in Monitis. The tests showed nice results which I saw in the Monitis dashboard:

 

Beside this, double-clicking on any line leads to an alternate view which shows additional data about the RabbitMQ state at that moment.

 

Eventually, you can also see a graphical representation of your data:

 

I built a rule for notification by using Monitis dashboard features and, satisfied, went to rest.

Perfect, let me summarize. I have obtained in a very short time the desired tool and moreover it can send me a notification any time of day about any troubling situation in my RabbitMQ engine.

Finally, I have uploaded the monitor I created into GitHub where you can find more details about it.

 

 

 

Share Now:del.icio.usDiggFacebookLinkedInBlinkListDZoneGoogle BookmarksRedditStumbleUponTwitterRSS

Read the original blog entry...

More Stories By Hovhannes Avoyan

Hovhannes Avoyan is the CEO of Monitis, Inc., a provider of on-demand systems management and monitoring software to 50,000 users spanning small businesses and Fortune 500 companies.

Prior to Monitis, he served as General Manager and Director of Development at prominent web portal Lycos Europe, where he grew the Lycos Armenia group from 30 people to over 200, making it the company's largest development center. Prior to Lycos, Avoyan was VP of Technology at Brience, Inc. (based in San Francisco and acquired by Syniverse), which delivered mobile internet content solutions to companies like Cisco, Ingram Micro, Washington Mutual, Wyndham Hotels , T-Mobile , and CNN. Prior to that, he served as the founder and CEO of CEDIT ltd., which was acquired by Brience. A 24 year veteran of the software industry, he also runs Sourcio cjsc, an IT consulting company and startup incubator specializing in web 2.0 products and open-source technologies.

Hovhannes is a senior lecturer at the American Univeristy of Armenia and has been a visiting lecturer at San Francisco State University. He is a graduate of Bertelsmann University.

@ThingsExpo Stories
SYS-CON Events announced today that Litmus Automation will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Litmus Automation’s vision is to provide a solution for companies that are in a rush to embrace the disruptive Internet of Things technology and leverage it for real business challenges. Litmus Automation simplifies the complexity of connected devices applications with Loop, a secure and scalable cloud platform.
While not quite mainstream yet, WebRTC is starting to gain ground with Carriers, Enterprises and Independent Software Vendors (ISV’s) alike. WebRTC makes it easy for developers to add audio and video communications into their applications by using Web browsers as their platform. But like any market, every customer engagement has unique requirements, as well as constraints. And of course, one size does not fit all. In her session at WebRTC Summit, Dr. Natasha Tamaskar, Vice President, Head of Cloud and Mobile Strategy at GENBAND, will explore what is needed to take a real time communications ...
In 2015, 4.9 billion connected "things" will be in use. By 2020, Gartner forecasts this amount to be 25 billion, a 410 percent increase in just five years. How will businesses handle this rapid growth of data? Hadoop will continue to improve its technology to meet business demands, by enabling businesses to access/analyze data in real time, when and where they need it. Cloudera's Chief Technologist, Eli Collins, will discuss how Big Data is keeping up with today's data demands and how in the future, data and analytics will be pervasive, embedded into every workflow, application and infra...
As Marc Andreessen says software is eating the world. Everything is rapidly moving toward being software-defined – from our phones and cars through our washing machines to the datacenter. However, there are larger challenges when implementing software defined on a larger scale - when building software defined infrastructure. In his session at 16th Cloud Expo, Boyan Ivanov, CEO of StorPool, will provide some practical insights on what, how and why when implementing "software-defined" in the datacenter.
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY., and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. MangoApps provides private all-in-one social intranets allowing workers to securely collaborate from anywhere in the world and from any device. Social, mobile, and easy to use. MangoApps has been named a "Market Leader" by Ovum Research and a "Cool Vendor" by Gartner...
SYS-CON Media announced today that @ThingsExpo Blog launched with 7,788 original stories. @ThingsExpo Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @ThingsExpo Blog can be bookmarked. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago.
The world's leading Cloud event, Cloud Expo has launched Microservices Journal on the SYS-CON.com portal, featuring over 19,000 original articles, news stories, features, and blog entries. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. Microservices Journal offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Follow new article posts on Twitter at @MicroservicesE
SYS-CON Events announced today that robomq.io will exhibit at SYS-CON's @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. robomq.io is an interoperable and composable platform that connects any device to any application. It helps systems integrators and the solution providers build new and innovative products and service for industries requiring monitoring or intelligence from devices and sensors.
Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 16th Cloud Expo at the Javits Center in New York June 9-11 will find fresh new content in a new track called PaaS | Containers & Microservices Containers are not being considered for the first time by the cloud community, but a current era of re-consideration has pushed them to the top of the cloud agenda. With the launch of Docker's initial release in March of 2013, interest was revved up several notches. Then late last...
Wearable technology was dominant at this year’s International Consumer Electronics Show (CES) , and MWC was no exception to this trend. New versions of favorites, such as the Samsung Gear (three new products were released: the Gear 2, the Gear 2 Neo and the Gear Fit), shared the limelight with new wearables like Pebble Time Steel (the new premium version of the company’s previously released smartwatch) and the LG Watch Urbane. The most dramatic difference at MWC was an emphasis on presenting wearables as fashion accessories and moving away from the original clunky technology associated with t...
So I guess we’ve officially entered a new era of lean and mean. I say this with the announcement of Ubuntu Snappy Core, “designed for lightweight cloud container hosts running Docker and for smart devices,” according to Canonical. “Snappy Ubuntu Core is the smallest Ubuntu available, designed for security and efficiency in devices or on the cloud.” This first version of Snappy Ubuntu Core features secure app containment and Docker 1.6 (1.5 in main release), is available on public clouds, and for ARM and x86 devices on several IoT boards. It’s a Trend! This announcement comes just as...
The only place to be June 9-11 is Cloud Expo & @ThingsExpo 2015 East at the Javits Center in New York City. Join us there as delegates from all over the world come to listen to and engage with speakers & sponsors from the leading Cloud Computing, IoT & Big Data companies. Cloud Expo & @ThingsExpo are the leading events covering the booming market of Cloud Computing, IoT & Big Data for the enterprise. Speakers from all over the world will be hand-picked for their ability to explore the economic strategies that utility/cloud computing provides. Whether public, private, or in a hybrid form, clo...
Internet of Things (IoT) will be a hybrid ecosystem of diverse devices and sensors collaborating with operational and enterprise systems to create the next big application. In their session at @ThingsExpo, Bramh Gupta, founder and CEO of robomq.io, and Fred Yatzeck, principal architect leading product development at robomq.io, will discuss how choosing the right middleware and integration strategy from the get-go will enable IoT solution developers to adapt and grow with the industry, while at the same time reduce Time to Market (TTM) by using plug and play capabilities offered by a robust I...
IoT is still a vague buzzword for many people. In his session at @ThingsExpo, Mike Kavis, Vice President & Principal Cloud Architect at Cloud Technology Partners, discussed the business value of IoT that goes far beyond the general public's perception that IoT is all about wearables and home consumer services. He also discussed how IoT is perceived by investors and how venture capitalist access this space. Other topics discussed were barriers to success, what is new, what is old, and what the future may hold. Mike Kavis is Vice President & Principal Cloud Architect at Cloud Technology Pa...
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? Join this panel of experts as they peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you’ll have no problem filling in your buzzword bingo cards.
SYS-CON Events announced today that CodeFutures, a leading supplier of database performance tools, has been named a “Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. CodeFutures is an independent software vendor focused on providing tools that deliver database performance tools that increase productivity during database development and increase database performance and scalability during production.
SYS-CON Events announced today the IoT Bootcamp – Jumpstart Your IoT Strategy, being held June 9–10, 2015, in conjunction with 16th Cloud Expo and Internet of @ThingsExpo at the Javits Center in New York City. This is your chance to jumpstart your IoT strategy. Combined with real-world scenarios and use cases, the IoT Bootcamp is not just based on presentations but includes hands-on demos and walkthroughs. We will introduce you to a variety of Do-It-Yourself IoT platforms including Arduino, Raspberry Pi, BeagleBone, Spark and Intel Edison. You will also get an overview of cloud technologies s...
SYS-CON Events announced today that Creative Business Solutions will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Creative Business Solutions is the top stocking authorized HP Renew Distributor in the U.S. Based out of Long Island, NY, Creative Business Solutions offers a one-stop shop for a diverse range of products including Proliant, Blade and Industry Standard Servers, Networking, Server Options and Care Packs. As a trusted supplier, CBS guarantees quality controlled stock levels thanks to an Auto...
SOA Software has changed its name to Akana. With roots in Web Services and SOA Governance, Akana has established itself as a leader in API Management and is expanding into cloud integration as an alternative to the traditional heavyweight enterprise service bus (ESB). The company recently announced that it achieved more than 90% year-over-year growth. As Akana, the company now addresses the evolution and diversification of SOA, unifying security, management, and DevOps across SOA, APIs, microservices, and more.
The list of ‘new paradigm’ technologies that now surrounds us appears to be at an all time high. From cloud computing and Big Data analytics to Bring Your Own Device (BYOD) and the Internet of Things (IoT), today we have to deal with what the industry likes to call ‘paradigm shifts’ at every level of IT. This is disruption; of course, we understand that – change is almost always disruptive.