Welcome!

Open Source Cloud Authors: Stefan Dietrich, Jeev Trika, Elizabeth White, Yeshim Deniz, Liz McMillan

Related Topics: Java IoT, Industrial IoT, Microservices Expo, Open Source Cloud, IoT User Interface, Apache

Java IoT: Blog Feed Post

Agile Architecture

The platform architecture defines common services that manage business delivery

The English language is well known for its subtlety. Sometimes it’s a delight, but on other occasions it can be very frustrating. If I use the term Gothic Architecture you will immediately understand I am describing a style of architecture that flourished in medieval times. And if like me you are interested in ecclesiastical architecture you will know that this style was used in many of the great cathedrals and churches across Europe, which were distinctive because of key architectural patterns that enabled great increases in height and internal light of the buildings without increasing the size of supporting pillars.

Now if I use the term Agile Architecture, what am I referring to? In today’s Agile world I would hazard a guess that most readers will think I am referring to the architecture techniques and tasks undertaken in the context of an Agile software development project, not the collection of patterns and practices that enable agile business systems. That is, an architecture that enables agility.

This potential for miscommunication is a core issue for enterprises. There is ample evidence that Agile Architecture is a primary contributor to business agility, yet we do not have a well understood architecture management system that integrates with Agile methods.

Let’s use an example readers may be familiar with. Amazon CEO Jeff Bezos famously [1]issued an edict that laid down some key architecture principles to Amazon development teams that I will summarize as:
· All teams will henceforth expose their data and functionality through service interfaces.
· Teams must communicate with each other through these interfaces. There will be no other form of interprocess communication allowed.
· It doesn't matter what technology they use.
· All service interfaces, without exception, must be designed from the ground up to be externalizable.
· No exceptions.

What Bezos did here was to lay down key business and technology architecture principles that you might reasonably conclude were central to the extraordinary level of business agility that we have seen demonstrated by Amazon.com, Inc. That widely circulated edict contained the foundations of the Amazon reference architecture.  

In the October 2004 CBDI Journal[2] we commented, “Two of the most successful and enduring dotcom start-ups, Amazon and eBay, now expose their core applications as Web Services. In doing so they have created a new class of platform that could have a profound impact on end-user organizations and IT vendors alike.”

And so the reference architecture became the enabler of growth and agility for the Amazon business, not we understand[3] as a grand plan, but through natural technological evolution. The services formed the platform that allowed the extraordinary expansion of the Amazon business that I would be certain not even Jeff Bezos imagined, back then in 2004. That is real business agility, and it was delivered by smart architecture backed up by clear policies and realized by agile processes.

Although Amazon has clearly evolved in pursuit of solutions to specific business opportunities and challenges, it’s also clear they have established a de facto architecture and architecture management system that guides the work of the many product delivery teams and ensures consistency of approach where it’s required. Let’s consider how an enterprise might establish a similar agile architecture management system.

A reference architecture articulates primary principles that are typically central to an entire enterprise. Principles should be focused on establishing the product and solution independent environment in which agility can be delivered and maintained, so they would be stable over time. We might refer to reference architecture as a Level 1 architecture perspective (L1) that exists purely as a set of models and guidelines.

Larger enterprises should explore the business value potential of platform based architecture as a mechanism to deliver cross enterprise consistency of core reference architecture behaviors and to enable closer integration with the wider ecosystem including customers, suppliers, end consumers etc. This is an extended management services platform which encapsulates the technology infrastructure and enables rapid delivery of business services.

The platform architecture defines common services that manage business delivery including security, life cycle management, change management, release management and operations, as well as catalogs, eCommerce, B2B, regulatory control and risk management, standardizing these key capabilities and reducing the footprint of business domain services. The platform will also manage important behaviors that deliver on specific business goals such as scalability and availability. For example, Amazon services are usually very fine grained, specifically to reduce the scope of each service in order to facilitate narrow focus SLAs and maximize scalability by reducing individual service complexity. We might refer to platform architecture as a Level 2 architecture perspective, engineered to be relatively stable in support of  large numbers of business services and consumers, but also engineered to evolve and respond rapidly to business and technology change. Not all enterprises will see business value in making their platform and business services available to their ecosystem, but some will.

Enterprises clearly vary considerably in their make up in terms of geographic and organizational, product and process standardization and differentiation, but typically there will be considerable potential for an inventory of shared assets that leverage agile architecture to support business agility. The assets may include:
· Common services, frameworks and components that are designed to deliver common behaviors to all parts of the enterprise. For example core services that establish genuinely enterprise wide services such as Customer, Ticket, eCommerce etc; services that deliver business value by standardizing common business services and processes.

· Configurable services, frameworks and components that are designed to provide common behaviors but are engineered to be customizable in local situations to accommodate many aspects of localization ranging from the simple – taxation, geography etc, to the complex – variant ordering patterns, variations in event and process sequence dictated by local de facto business practices. Configurable services may provide business value simply by providing reusable components, or they may establish a common core of business process and information that establishes common reporting and regulatory control in a local context, or both. Configurable services may also be an important time to market strategy for service providers who customize their services for each client or customer group.

· Information architecture and services. Establishing a coherent approach to information is commonly a major issue for large enterprises and this architecture level defines an integrated approach for structured and unstructured (big) data, transactional and reference, enterprise reporting and regulatory control and so on.

Common and Configurable assets together with the Information Architecture might form a Level 3 architecture perspective and be widely applicable across a large, distributed enterprise.  

We then have two further levels which are closely related, Family Architecture and Product Line Architecture. Whilst many architects chose to view Family and Product Line as synonyms, I recommend that they are kept separate. A Family architecture is a domain framework that is much more specialized that L3 assets that would be applicable on a broader basis. The Family architecture establishes core business (domain) services and possibly other artifacts specific to the domain, where the domain is likely to be a subject area or a cluster of major types. For example Customer, Supply Chain, Manufacturing, Risk etc. Families are also commonly acquired products.

In contrast Product Line architecture is what it says – it’s the architecture for a product offering. The product is an offering that has direct relationship to end customer revenue and usually continuity of purpose over multiple releases. Although from a narrow technical perspective the Product and Family architectures might be similar, the way a product is managed must mirror the business product life cycle. Family architectures may therefore be engineered for stability, whereas, depending on the industry sector, product line architectures may be engineered for maximum agility and minimum response time.  

Finally we have the Solution architecture level, the architecture specific to solution project delivery, where the focus is on feature architecture and integrating solution architecture with the Level 1 to 5 architecture perspectives. It’s important to note that where product line architecture is used, then this may subsume the Solution architecture.

These six architecture levels provide us with a nomenclature for agile architecture that will be central to managing agility into the delivered product/solution. The architecture perspective guides the structure of programs and projects and the incorporation of architecture and reuse goals into delivery charters. The architecture also provides traceability and governance over realization of core architecture principles.

The question of how Agile Architecture integrates with Agile delivery is likely to prove contentious because architecture introduces a form of direction that contradicts Agile concepts. Yet the lessons from Amazon are insightful. The most senior business management need to be fully engaged and actively leading the development of architectural direction. Further in large enterprises customer project demand needs to be managed and aligned with business strategy and architectural direction.

There’s no reason why these Demand and Definition processes shouldn’t adopt Agile concepts, notably cross functional teams, time boxes and backlogs. The outcomes should be excellent visibility and traceability of key strategies and policies that provide real clarity of purpose for projects, that will increase the probability of success. In a typical large enterprise use of existing (or well understood) organizational concepts, adjusted to use aspects of Agile methods as discussed, will meet less organizational resistance. For example:  

1. Architecture Review Board (ARB) or equivalent, a cross functional team (senior representatives of business, product management, architecture and delivery), that provide direction and funding to all architecture development.
2. Design Authority (DA), also a cross functional team (domain specific expert level representatives of business, product management, architecture and delivery), that transform raw customer demand stream into project charters and manage the portfolio view. It is the DA that takes responsibility for aggregating and decomposing customer and strategic demand, chartering Common, Product Line and Family architecture, typically as integral elements of delivery projects, which can demonstrate business value.
3. Investigatory architecture projects – short duration projects that validate assumptions prior to chartering composite architecture/delivery projects. Sometimes carried out as part of a Definition Phase activity concurrent with outline requirements and knowledge discovery. Using patterns as a mechanism to increase consistency of architecture decisions and communicate them to delivery projects at sensible level of detail that is useful to delivery teams.  Recommend includes delivery team members as appropriate.
Note this is a recursive model, and the process may executed at enterprise and program level.

You may ask where Enterprise Architecture is in this. The answer is that enterprise architecture is a role and responsibility that must coordinate and govern all levels of architecture. Enterprise Architects are most likely to be assigned to a specific architecture perspective level. The notion of, “one architecture to rule them all” really doesn’t exist.
Each enterprise should develop its own architecture management approach, and integrate this into an end to end architecture, delivery and governance process. The term Agile Architecture should be used to describe and deliver architecture that facilitates the agile business by compliance with reference, platform and other architectures that facilitate evolution, customization and plug and play. Faster cycle time and quality outcomes are then a function of both the reusable patterns and parts available for assembly and the Agile delivery process.  

In medieval times the builders of the Gothic cathedrals didn’t start their designs from scratch. But equally they didn’t have finely detailed (ivory tower) plans – the technology didn’t exist to support that. Master builders moved from city to city bringing their proven architecture in their heads, often together with experienced craftsmen, to new projects. Craftsmen and master builders together tried out new designs and gradually evolved core patterns such as the flying buttress, which became standard components in cathedrals across Europe. Sometimes the great buildings fell down during construction and the builders had to adapt the architecture and try again. They were truly early adopters of Agile methods as they combined architecture and build in what clearly was from time to time an empirical delivery approach, but they also had their equivalent of a reference architecture and patterns that enabled systematic reuse of proven designs. Of course their delivery cycle time was a little longer than today’s Agile project!


Talk to Everware-CBDIabout the Agile Enterprise Workshop. This is currently available as an in-house, intensive workshop. Public scheduled classes will hopefully follow next year.




[1] Amazon and eBay Web Services, The New Enterprise Applications? By Lawrence Wilkes, CBDI Journal October 2004


[2] Inadvertently published by Steve Yegge, 2011, in a comparison of Google and Amazon practices. http://upalc.com/google-amazon.php

[3] Werner Vogels, 2006, SOA creates order out of chaos @ Amazon, Rich Seeley, Search SOA

Read the original blog entry...

More Stories By David Sprott

David Sprott is a consultant, researcher and educator specializing in service oriented architecture, application modernization and cloud computing. Since 1997 David founded and led the well known think tank CBDI Forum providing unique research and guidance around loose coupled architecture, technologies and practices to F5000 companies and governments worldwide. As CEO of Everware-CBDI International a UK based corporation, he directs the global research and international consulting operations of the leading independent advisors on Service Oriented Application Modernization.

@ThingsExpo Stories
The IoTs will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm and share the must-have mindsets for removing complexity from the development proc...
Customer experience has become a competitive differentiator for companies, and it’s imperative that brands seamlessly connect the customer journey across all platforms. With the continued explosion of IoT, join us for a look at how to build a winning digital foundation in the connected era – today and in the future. In his session at @ThingsExpo, Chris Nguyen, Group Product Marketing Manager at Adobe, will discuss how to successfully leverage mobile, rapidly deploy content, capture real-time d...
SYS-CON Events announced today that ContentMX, the marketing technology and services company with a singular mission to increase engagement and drive more conversations for enterprise, channel and SMB technology marketers, has been named “Sponsor & Exhibitor Lounge Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York. “CloudExpo is a great opportunity to start a conversation with new prospects, but what happens after the...
What a difference a year makes. Organizations aren’t just talking about IoT possibilities, it is now baked into their core business strategy. With IoT, billions of devices generating data from different companies on different networks around the globe need to interact. From efficiency to better customer insights to completely new business models, IoT will turn traditional business models upside down. In the new customer-centric age, the key to success is delivering critical services and apps wit...
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, will provide an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life ...
SYS-CON Events announced today that BMC Software has been named "Siver Sponsor" of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. BMC is a global leader in innovative software solutions that help businesses transform into digital enterprises for the ultimate competitive advantage. BMC Digital Enterprise Management is a set of innovative IT solutions designed to make digital business fast, seamless, and optimized from mainframe to mo...
SYS-CON Events announced today that MobiDev will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business managers to a full-scale mobile software company with over 200 develope...
SoftLayer operates a global cloud infrastructure platform built for Internet scale. With a global footprint of data centers and network points of presence, SoftLayer provides infrastructure as a service to leading-edge customers ranging from Web startups to global enterprises. SoftLayer's modular architecture, full-featured API, and sophisticated automation provide unparalleled performance and control. Its flexible unified platform seamlessly spans physical and virtual devices linked via a world...
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
As cloud and storage projections continue to rise, the number of organizations moving to the cloud is escalating and it is clear cloud storage is here to stay. However, is it secure? Data is the lifeblood for government entities, countries, cloud service providers and enterprises alike and losing or exposing that data can have disastrous results. There are new concepts for data storage on the horizon that will deliver secure solutions for storing and moving sensitive data around the world. ...
SYS-CON Events announced today TechTarget has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. TechTarget is the Web’s leading destination for serious technology buyers researching and making enterprise technology decisions. Its extensive global networ...
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. MangoApps provides modern company intranets and team collaboration software, allowing workers to stay connected and productive from anywhere in the world and from any device. For more information, please visit https://www.mangoapps.com/.
The essence of data analysis involves setting up data pipelines that consist of several operations that are chained together – starting from data collection, data quality checks, data integration, data analysis and data visualization (including the setting up of interaction paths in that visualization). In our opinion, the challenges stem from the technology diversity at each stage of the data pipeline as well as the lack of process around the analysis.
Designing IoT applications is complex, but deploying them in a scalable fashion is even more complex. A scalable, API first IaaS cloud is a good start, but in order to understand the various components specific to deploying IoT applications, one needs to understand the architecture of these applications and figure out how to scale these components independently. In his session at @ThingsExpo, Nara Rajagopalan is CEO of Accelerite, will discuss the fundamental architecture of IoT applications, ...
SYS-CON Events announced today that Tintri Inc., a leading producer of VM-aware storage (VAS) for virtualization and cloud environments, will exhibit at the 18th International CloudExpo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
In his session at 18th Cloud Expo, Bruce Swann, Senior Product Marketing Manager at Adobe, will discuss how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects). Bruce Swann has more than 15 years of experience working with digital marketing disciplines like web analytics, social med...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...