|By Kamyar Emami||
|January 14, 2013 07:00 AM EST||
The industrial revolution continues - starting with the steam engines of the 18th century, continuing with large-scale steel production, oil exploitation, electrical and photographic innovations of the 19th century, and moving on to the transportation, communications, computation and electronics of the 20th century. It is still early in the 21st century, but we can safely say software has become the engine that feeds the industrial, economic, medical, and gradually the political issues of our existence. The only way to satisfy the demand for the volume and complexity of the software that is needed to keep our world moving is to maximally share and reuse code within and across application domains.
Open Source Software (OSS) is the epitome of code reuse, enabling complex applications to be realized rapidly, economically and safely. Probably the largest collaborative endeavor in human kind to date, open source feeds on itself. Independent studies have converged on the fact that open source is everywhere, and depending on the source of the study, 80-100% of all software organizations now use open source software in their products or operations.
There is an implicit understanding that good developers do not write code from scratch any more. Rather, they can adapt a piece of existing code to furnish a desired function. Use of off-the-shelf code such as OSS brings the usual due diligence and precautions associated with deployment of any third-party content within an organization. The pedigree of the code, its ownership attributes, and the rules around the use of open source code (typically captured in a license document) govern its introduction within an organization and its suitability for an end-target use.
Open source software brings with it an unusual set of ownership issues. Unlike other commodities, open source code can be brought into an organization freely. While anything that is purchased in a transaction has implied ownership, ownership and usage of OSS can be confusing to many developers or organizations. Generally, the copyright ownership of OSS always stays with the creator of the open source code. The OSS copyright owner creates and communicates a license that explicitly sets out the rules governing the use of that open source software.
Continuous Versus One-Time OSS Assessment
Detecting and complying with OSS code in a software project has become an important part of a quality and governance process in organizations that create or consume software.
Product quality considerations and standards require that a recorded knowledge of all the third-party components within a product be maintained at all times. These records should also include attributes such as pedigree, defect history and improvements over time, potential vulnerabilities, and code propagation within the organization. These records can be best maintained, not by a one-time examination of the organization's code portfolio, but through an ongoing and structured third-party and open source software adoption process. The practice of creating and updating the records automatically as development proceeds ensures ongoing compliance with the requirements of a quality organization.
As opposed to the continuous recordkeeping requirements of a quality process, a software audit is a one-time activity targeted at providing insight into the intellectual property (IP) ownership or IP rights, in anticipation of a transaction such as an M&A or a product shipment to market.
Open source software audits, generally carried out by an external body, involves an examination of a software portfolio in order to detect OSS and third-party code within that portfolio. The result of the audit is a report which highlights open source and third party components and their attributes. At a high level, statistics such as the names of any public-domain software packages and whether they are used in a modified or unmodified format, composition of the license mix, copyrights, vulnerabilities, languages, and open source lines of code are provided. At a more detailed level, specific open source or proprietary packages that were discovered, their license attributes, links to resources that contain additional information, text of the licenses, copyrights, and know security vulnerabilities associated with the components of the software are provided.
An audit process would highlight code that is specifically copyrighted, but for which no license is offered or mentioned. These cases are one of the challenging aspects in establishing IP ownership, as the copyright owner must be contacted for explicit permission to use their code. Also, any code that is not in the public domain and has no identifying information, such as headers, must be highlighted as requiring further investigation.
The Science: Detecting Third-Party Packages in a Portfolio
Once all OSS and other third-party packages are identified and a software Bill of Materials (BoM) is available, then the list can be examined for properties such as licenses, known deficiencies, security vulnerabilities, various obligations associated with their use, and functions such as encryption that could restrict its use in certain markets.
A number of methods can be used to identify open source and commercial software within a software portfolio. A short list of these methods will include the following.
- Records by developers: Any records maintained by developers and development managers will assist the process.
- Information held within a file, or folder: A quality-development practice is to include a header on every file that holds information about the software package, organization, copyright and license associated with the file or the package. Often binaries also include identifying information, such as copyright owner, project name, and the license pointers within the file. Another quality practice is to include licensing or other information about a software package within the package.
- File/folder names and paths: These could be additional indicators of the presence of a known public domain or commercial software.
- Similarity with public-domain software: Any similarity between a software file, or portions of a software file, and a file in public domain could accurately indicate the presence of OSS in a portfolio. A full-file code similarity to a public domain file would indicate unmodified use of the OSS software. A partial code match would indicate use of OSS in a modified form. This is significant because many open source licenses trigger different obligations based on how the file is used.
There are hundreds of thousands of public-domain projects accessible to developers. When you consider that an OSS project can have multiple versions in the public domain and each package can consist of anything between two and 200,000 files, we gain an appreciation for the task involved in this method. Manual identification of code similarity to millions of files is obviously impractical. Only intelligent automated solutions can go through a software portfolio and examine similarity between each and every file in that portfolio and software files in public domain.
The Art: Reading Between the Lines
The methods described above could theoretically provide insight into composition of a code portfolio. However, those methods alone are not sufficient to reveal an accurate view of the code composition.
- Manual records are the least reliable method as third-party content is often brought into a project without registering a record. Also, in today's typical development environment it is very difficult to guarantee access to the original developer's piece of software.
- File header information is not necessarily an accurate representation of the file pedigree and license. These can be changed by a developer, or automatically as in Linux kernel header files that were used in Android packages.
- Almost all OSS uses OSS. For example, there are more than 65,000 instances of commons.logging (a popular Apache logging layer), and more than 50,000 instances each of Log4j (another Apache logging utility for Java) and JUnit (a popular testing framework for Java) code in various public-domain projects. This leads to OSS project and license nesting complexities and contributes to challenges in correctly identifying the OSS packages within a portfolio. It is critical to detect the genesis OSS project and the version of the original and framing OSS projects. Practical examples of this challenge are:
- Open source packages that use other OSS but do not maintain or propagate the original OSS license. This action may not be legitimate depending on the original OSS license and the license of the derived OSS package and can lead to erroneous conclusions about the quality as well as usage obligations associated with the organization's software.
- Open source packages that change the license in subsequent versions such as moving from one version of a license. For example, GPLv2 to GPLv3 on a new release of the OSS project or moving from one license to another compatible license.
- Projects where the OSS file or folder name is modified. This happens regularly on libraries (those with .jar or .bin extensions), and less frequently on source code files.
- Seemingly known but non-existent licenses mentioned within a file or folder. A prime example is an OSS package that claims it's released under GPL. There is no GPL, only versioned GPL such as GPL v1, v2 licenses are available.
The art of OSS audit activity and license management relies on a clear understanding of the open source software community, open source packages, open source licenses, and development practices. This understanding comes with both academic knowledge as well as experience in scanning, reviewing, and auditing hundreds of software portfolios. Automated solutions that combine the science of scanning and license management with empirical methods that embody the art of open source package detection and license discovery can significantly speed up the discovery and management process and minimize, although not eliminate, the human involvement factor.
Rapid software development is necessary for sustaining the pace of innovation needed in today's world and the use of OSS is perhaps the best way to maintain this pace. While most organizations are now using open source to their advantage, they must avoid potential complications that OSS license obligations present. The complexity that OSS licenses present makes it almost impossible to manage obligations manually. This is where automated solutions come in. Conducting a one-time audit, or preferably, having a continuous process in place to automatically detect OSS licenses and their obligations is the best way to protect your organization from risk while ensuring continuous innovation.
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be.
Sep. 3, 2015 07:15 PM EDT Reads: 116
Containers are not new, but renewed commitments to performance, flexibility, and agility have propelled them to the top of the agenda today. By working without the need for virtualization and its overhead, containers are seen as the perfect way to deploy apps and services across multiple clouds. Containers can handle anything from file types to operating systems and services, including microservices. What are microservices? Unlike what the name implies, microservices are not necessarily small, but are focused on specific tasks. The ability for developers to deploy multiple containers – thous...
Sep. 3, 2015 06:00 PM EDT Reads: 146
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome,” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Sep. 3, 2015 04:30 PM EDT Reads: 428
The 3rd International WebRTC Summit, to be held Nov. 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 15th International Cloud Expo, 6th International Big Data Expo, 3rd International DevOps Summit and 2nd Internet of @ThingsExpo. WebRTC (Web-based Real-Time Communication) is an open source project supported by Google, Mozilla and Opera that aims to enable bro...
Sep. 3, 2015 03:00 PM EDT Reads: 1,605
As more and more data is generated from a variety of connected devices, the need to get insights from this data and predict future behavior and trends is increasingly essential for businesses. Real-time stream processing is needed in a variety of different industries such as Manufacturing, Oil and Gas, Automobile, Finance, Online Retail, Smart Grids, and Healthcare. Azure Stream Analytics is a fully managed distributed stream computation service that provides low latency, scalable processing of streaming data in the cloud with an enterprise grade SLA. It features built-in integration with Azur...
Sep. 3, 2015 02:45 PM EDT Reads: 386
With the proliferation of connected devices underpinning new Internet of Things systems, Brandon Schulz, Director of Luxoft IoT – Retail, will be looking at the transformation of the retail customer experience in brick and mortar stores in his session at @ThingsExpo. Questions he will address include: Will beacons drop to the wayside like QR codes, or be a proximity-based profit driver? How will the customer experience change in stores of all types when everything can be instrumented and analyzed? As an area of investment, how might a retail company move towards an innovation methodolo...
Sep. 3, 2015 02:30 PM EDT Reads: 509
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
Sep. 3, 2015 02:30 PM EDT Reads: 962
SYS-CON Events announced today the Containers & Microservices Bootcamp, being held November 3-4, 2015, in conjunction with 17th Cloud Expo, @ThingsExpo, and @DevOpsSummit at the Santa Clara Convention Center in Santa Clara, CA. This is your chance to get started with the latest technology in the industry. Combined with real-world scenarios and use cases, the Containers and Microservices Bootcamp, led by Janakiram MSV, a Microsoft Regional Director, will include presentations as well as hands-on demos and comprehensive walkthroughs.
Sep. 3, 2015 02:15 PM EDT Reads: 414
Contrary to mainstream media attention, the multiple possibilities of how consumer IoT will transform our everyday lives aren’t the only angle of this headline-gaining trend. There’s a huge opportunity for “industrial IoT” and “Smart Cities” to impact the world in the same capacity – especially during critical situations. For example, a community water dam that needs to release water can leverage embedded critical communications logic to alert the appropriate individuals, on the right device, as soon as they are needed to take action.
Sep. 3, 2015 01:30 PM EDT
WebRTC services have already permeated corporate communications in the form of videoconferencing solutions. However, WebRTC has the potential of going beyond and catalyzing a new class of services providing more than calls with capabilities such as mass-scale real-time media broadcasting, enriched and augmented video, person-to-machine and machine-to-machine communications. In his session at @ThingsExpo, Luis Lopez, CEO of Kurento, will introduce the technologies required for implementing these ideas and some early experiments performed in the Kurento open source software community in areas ...
Sep. 3, 2015 01:15 PM EDT Reads: 105
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies leverage disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 17th Cloud Expo, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advanced analytics, and DevOps to advance innovation and increase agility. Specializing in designing, imple...
Sep. 3, 2015 01:00 PM EDT Reads: 367
Consumer IoT applications provide data about the user that just doesn’t exist in traditional PC or mobile web applications. This rich data, or “context,” enables the highly personalized consumer experiences that characterize many consumer IoT apps. This same data is also providing brands with unprecedented insight into how their connected products are being used, while, at the same time, powering highly targeted engagement and marketing opportunities. In his session at @ThingsExpo, Nathan Treloar, President and COO of Bebaio, will explore examples of brands transforming their businesses by t...
Sep. 3, 2015 12:30 PM EDT Reads: 286
In his session at @ThingsExpo, Lee Williams, a producer of the first smartphones and tablets, will talk about how he is now applying his experience in mobile technology to the design and development of the next generation of Environmental and Sustainability Services at ETwater. He will explain how M2M controllers work through wirelessly connected remote controls; and specifically delve into a retrofit option that reverse-engineers control codes of existing conventional controller systems so they don't have to be replaced and are instantly converted to become smart, connected devices.
Sep. 3, 2015 12:00 PM EDT Reads: 255
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts, GM of Platform at FinancialForce.com, will discuss the value of business applications on wearable ...
Sep. 3, 2015 10:45 AM EDT
SYS-CON Events announced today that Micron Technology, Inc., a global leader in advanced semiconductor systems, will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Micron’s broad portfolio of high-performance memory technologies – including DRAM, NAND and NOR Flash – is the basis for solid state drives, modules, multichip packages and other system solutions. Backed by more than 35 years of technology leadership, Micron's memory solutions enable the world's most innovative computing, consumer,...
Sep. 3, 2015 10:00 AM EDT Reads: 284
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS – software, platform, and infrastructure as a service.
Sep. 3, 2015 10:00 AM EDT Reads: 1,603
As more intelligent IoT applications shift into gear, they’re merging into the ever-increasing traffic flow of the Internet. It won’t be long before we experience bottlenecks, as IoT traffic peaks during rush hours. Organizations that are unprepared will find themselves by the side of the road unable to cross back into the fast lane. As billions of new devices begin to communicate and exchange data – will your infrastructure be scalable enough to handle this new interconnected world?
Sep. 3, 2015 09:30 AM EDT Reads: 203
While many app developers are comfortable building apps for the smartphone, there is a whole new world out there. In his session at @ThingsExpo, Narayan Sainaney, Co-founder and CTO of Mojio, will discuss how the business case for connected car apps is growing and, with open platform companies having already done the heavy lifting, there really is no barrier to entry.
Sep. 3, 2015 09:30 AM EDT Reads: 212
SYS-CON Events announced today that the "Second Containers & Microservices Expo" will take place November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities.
Sep. 3, 2015 09:30 AM EDT Reads: 651
Manufacturing connected IoT versions of traditional products requires more than multiple deep technology skills. It also requires a shift in mindset, to realize that connected, sensor-enabled “things” act more like services than what we usually think of as products. In his session at @ThingsExpo, David Friedman, CEO and co-founder of Ayla Networks, will discuss how when sensors start generating detailed real-world data about products and how they’re being used, smart manufacturers can use the data to create additional revenue streams, such as improved warranties or premium features. Or slash...
Sep. 3, 2015 09:15 AM EDT