Welcome!

Open Source Cloud Authors: Liz McMillan, Pat Romanski, Yeshim Deniz, Elizabeth White, Zakia Bouachraoui

Related Topics: Open Source Cloud, Microservices Expo

Open Source Cloud: Article

Case Study: Open Source + Business Intelligence

A marriage made for data-driven businesses

Data-driven businesses are facing some tough challenges in today's rapidly changing information landscape. As decision cycles continue to shrink, companies need to act on information within hours and minutes rather than weeks and days. At the same time, the volume of data that needs to be analyzed is growing exponentially. Business intelligence (BI) approaches that might have made sense a decade or even five years ago may no longer be the best fit for organizations that must quickly and affordably make sense of terabytes of incoming data that shows no sign of slowing down.

For my company, MX Force, speedy data analysis is not simply a "nice to have," it's critical to our business. As a cloud-based provider of email security for organizations of all sizes, we need to identify the origins of spam, viruses and other potential threats for our clients, fast. But as our business has grown, so has the volume of email log data that we must store, filter, search, analyze and report on. Recently, we were challenged to find a database that could reliably enable quick and efficient ad-hoc queries on up to a year's worth of email log data. Our staff uses this data to analyze and report on statistical information, and we also give our clients the ability to query their own logs to diagnose mail delivery issues. It was important to find a database that could deliver the high performance we required, but affordability and ease of administration were also of vital concern. These considerations prompted us to seek an open source solution.

Open Source Meets Business Intelligence
MX Force uses a number of open source tools within our organization. The low cost of open source is one reason for this, but flexibility is another important driver. Because open source projects are community-driven, users can tweak, customize and tinker with the software as much as they like. This is a big advantage when it comes to business intelligence, as data analysis requirements can change quickly, and you don't want to have to wait weeks or months to get a new query set up or to change the parameters of those that are already running. MX Force was already using MySQL in our business, so we decided to try Infobright's open source analytic database, ICE (Infobright Community Edition.) ICE combines a columnar database with innovative compression and self-tuning capabilities that eliminate the need to create indexes, partition data or do any manual intervention to achieve fast response for queries and reports. The software is built on MySQL, so for us there was a very small implementation and training curve - ICE uses the same familiar MySQL interface. The fact that ICE is an open source analytic solution presented us with several key benefits:

  1. Deployment speed: The time from download and installation to first production use was just three weeks.
  2. Affordability: Many of the proprietary commercial BI solutions available today require custom configuration, expensive licensing agreements and equally expensive hardware to support and run it. Not only was ICE free to install, we could also run the software on inexpensive commodity servers, eliminating the need to invest in high performance servers and storage arrays. (Our entire workload is supported by a single quad-core server.)
  3. Simplicity and flexibility: Because ICE is open and standards-based, we can quickly make changes as needed without requiring extensive IT assistance. In addition, it's often a lot simpler to make fixes or upgrade an open source solution because an entire community contributes their expertise to fixing bugs and making improvements. With proprietary software, users have to wait for issues to be addressed by the vendor, which can take much longer.

MX Force is currently using ICE to quickly isolate mail flow problems and trends. In our experience, using a free, open source product has not in any way involved a compromise on performance or capabilities. We are achieving 10:1 data compression, which saves on storage costs and boosts performance. Most statistical queries render results in less than five seconds. Ongoing administration is simple. The net result is that the product delivers the fast query performance and reporting functionality we needed, at an incredibly low cost for hardware and ongoing maintenance.

Look, then Leap
Interested in giving open source a try for your BI and analytic efforts? There are a number of compelling benefits to doing so, but as with any type of software, it's also important to look before you leap. Evaluation and testing considerations are no different than they would be for licensed software - you want to make sure the solution has the features and capabilities most relevant to your business. Also, there's a difference between open source projects that are at a very early and experimental stage and software that is well established and has a vibrant and involved community behind it, strong vendor support, or both. Investigate the support offered for the solution under consideration. How often are new features added? Are bug fixes made in a timely manner? Is there useful and accurate supporting documentation?

With ICE, we were certainly attracted by the many resources and significant participation of both Infobright and the user community. We also knew there was a commercial version available if we decided we needed the additional functionality it offered or a formal support contract. For companies just jumping in to the open source arena, it's best to avoid tools that haven't yet cultivated a strong following. But even if you do make a mistake, the low (and usually free) cost of open source means that there's minimal risk.

The BI requirements of today's data-driven businesses demand speed, simplicity and affordability. As open source solutions continue to mature, it's worth looking at projects that are focused on analytics, BI and other data management activities. The more nimble and flexible approach embodied by open source may just be the best fit for addressing the many information management challenges driven by data growth and complexity.

More Stories By Mike Makowski

Mike Makowski is CTO of MX Force, a leading provider of email security in the cloud and member of Infobright’s Customer Advisory Council. More information about MX Force can be found at http://www.mxforce.com/

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
In this Women in Technology Power Panel at 15th Cloud Expo, moderated by Anne Plese, Senior Consultant, Cloud Product Marketing at Verizon Enterprise, Esmeralda Swartz, CMO at MetraTech; Evelyn de Souza, Data Privacy and Compliance Strategy Leader at Cisco Systems; Seema Jethani, Director of Product Management at Basho Technologies; Victoria Livschitz, CEO of Qubell Inc.; Anne Hungate, Senior Director of Software Quality at DIRECTV, discussed what path they took to find their spot within the tec...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and G...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
DXWorldEXPO LLC announced today that "IoT Now" was named media sponsor of CloudEXPO | DXWorldEXPO 2018 New York, which will take place on November 11-13, 2018 in New York City, NY. IoT Now explores the evolving opportunities and challenges facing CSPs, and it passes on some lessons learned from those who have taken the first steps in next-gen IoT services.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...