Linux is the fastest-growing embedded operating environment in the world today. It's quickly becoming the single largest operating system platform for embedded computing. As a result, many technology managers must come to grips with the complexity and the dynamics of Open Source software in general and Linux evolution in particular. Particular questions and concerns arise in the areas of compatibility, the role and nature of different versions of Linux (the "forking issue"), and the technology advancement process itself. The history of incompatible proprietary versions of Unix contributes significantly to these concerns. It's critical for the continued successful adoption of Linux that these concerns be assessed and understood. The nature of Linux as Open Source dramatically changes not just the specifics of these concerns, but relative to the history of Unix, changes even the nature of the concerns themselves.

The Nature of Linux Innovation
Linux is Open Source. What does this mean (in terms that are significant to compatibility and forking)? There are several critical facts:

• Linux is formally a trademark, owned by Linus Torvalds. The trademark is associated with a body of source code hosted at the www.kernel.org/ Web site.
• Being Open Source, anyone is free to procure a copy of Linux source code from kernel.org, and modify it.
• Those who do so are encouraged to submit their modifications back to the maintainers of the Linux source code. These submissions may be accepted quickly, may enter a short or long period of discussion and possible evolution before being accepted, or may be rejected.
• Through this process, Linux advances as a technology.
• Independent bodies of source code derived from the kernel.org source code base are routinely called Linux, even when modified. Mr. Torvalds has not campaigned to restrict the use of the Linux trademark to the formal code that resides at kernel.org.

Without this "external to the kernel.org" source code modification process, Linux could not and would not advance. As with any software development program, the "master copy" is only modified when the changes, made separately, are to a greater or lesser degree "complete" and "validated." Unlike proprietary software where an exclusive few are allowed to participate in the innovation cycle, with Linux all software engineers are encouraged to get involved, innovate, and contribute back. This independent innovation activity is happening continuously, everyday, by people working in an individual ("amateur") capacity, and by technology companies motivated by business needs and opportunities.

Market Requirements
As with all vibrant system software technologies, Linux continues to rapidly advance in capabilities to meet new and increased requirements. New requirements arise from multiple sources. One key source is hardware advances. As new features are developed and made available in commercial silicon, Linux must advance to enable and take advantage of that hardware feature. A few recent examples of this include 64 bit processors, on-board security features, multi-core processors with various operating modes, and, of course, extensive new and enhanced I/O capabilities.

A second key source of new requirements for Linux is the Linux's encroaching into new markets or application domains. The general adoption of Linux into embedded computing environments since the late 1990s has created a tremendous number of technical challenges for Linux. These include advanced high-availability requirements for the telecommunications market, improvements in flash memory utilization and real-time performance for multiple markets, boot time and power management improvements for the consumer products and mobile handset markets, and many more.

Linux Latency and Linux Stability
The source code base at kernel.org is managed in a way that strives to balance stability and rapid innovation. The particular mechanics used have evolved over recent years, and no doubt will continue to evolve. What has been constant is that, overall, innovation is the primary focus rather than stability. As a result, those familiar with the daily and weekly level of evolution of the kernel.org software are aware that the leading-edge source base can't always be expected to be in a highly reliable and robust state relative to typical commercial requirements for stability.

There is also a natural latency in any code modification submitted to the kernel.org process. Small "obvious" corrections may only take a few days to move through the process and become integrated into the Open Source code base. Larger corrections and significant incremental value components can take weeks, months, or even years of assessment, debate, refinement, usage feedback and, in general, understanding, before being formally accepted and integrated into the kernel.org source base.

There are several critical pragmatic outcomes of these facts. First, basing a commercial embedded system product development program on the kernel.org source base as "the" source base for the project (i.e., literally utilizing no local copy) is somewhere between extremely risky and outright impossible. A single, small, required change in code would generate the need to "re-source" from kernel.org, and in doing that potentially large amounts of changes would be picked up. Doing this late in a development program could have dramatically deleterious effects on the product program. For this reason, everyone doing serious product development uses a private copy of the Linux source base. A stable, minimally changing set of product source code that's under the absolute change control of the product team is a fundamental requirement for commercial software development, even when major components of that product are based on Open Source code.

Similarly, anytime an advanced feature is required that isn't already extant in the kernel.org software, it's necessary to procure or develop a derivative version of Linux oneself that includes the required feature. Often such features exist as Open Source component software (typically structured as patches for the current kernel.org source code). Such features are frequently included in commercial products, such as MontaVista Linux. Sometimes, the features require new development. It can be argued that submitting such features back to the kernel.org source base is in the best interests of the developing organization (but that's the subject of a different article). What's critical to understand here is that any such submission and re-sourcing of the kernel.org software with the advanced feature included is rarely a viable development path for a commercial product development program. The delay and instability risks dictate that development, stabilization, and deployment using a privately derived set of source code is the only viable approach.