Agility

Software applications are seldom built for the sort of general purpose uses that a computer is suitable for. Despite our ingenuity in building computer chips that can run everything from aircraft to video games, the software itself is good for only one kind of activity or another. The result is that all too often software that is anything but “soft” – it is very brittle and difficult to change.

In chapter8, we emphasized that an SOA addresses this long-standing problem by treating software quality – and in particular agility – as an integral part of software architecture. It is important to note that this does not mean formalizing the development process, as in the “engineering” approach to software quality, exemplified by the CMMI, the established approach to formalizing the processes of creating and managing software.

The fact is that when we build software to meet specific, predefined requirements, it almost always falls short of meeting the true requirements for that software, primarily because the actual requirements for software are typically in a state of constant flux. Software must be agile enough to meet as yet undefined requirements at some point in the future.

In this chapter, we therefore focus on how to design agile services. This takes us to the heart of SOA: modeling techniques that help get the right set of services.

Sam Higgins is now with Forrester Research, Inc. as a senior analyst. Formerly he was a director of Encode Services, an Australian information and communications technology consulting firm, and managed application architecture direction for the Innovation and Planning unit of Queensland Transport's Information Services Branch. He has over twelve years' experience in both traditional and model-driven environments, and is a popular international speaker. Sam was a driving force behind Queensland Transport's adoption of the Java 2 Enterprise Edition (J2EE) platform and subsequent Web services implementations.

Paul McRae is the application architect for the Innovation and Planning unit of Queensland Transport's Information Services Branch. He has almost twenty years' experience in the IT industry, was a key player in the introduction of an SOA within Queensland Transport, and was instrumental in developing Queensland Transport's CBD initiatives, J2EE applications, enterprise architecture, and Web services.

In this chapter Sam and Paul share their experiences in Queensland Transport's transition to service orientation.

Background

Queensland is Australia's north-eastern state, covering nearly 1.7 million km2, and home to approximately 3.6 million people. Known as the “Smart State,” Queensland is a center of significant leading-edge technology developments. Queensland Transport is a department within the State Government of Queensland.

Queensland Transport has a budget of approximately A$1.3 billion and over 4,700 staff throughout the state, and is responsible specifically for the regulation of land, sea, and aviation transport in Queensland.

Hermann Schlamann is a senior architect in the architecture group of Credit Suisse (CS), having worked as a consultant in the banking industry in Switzerland since the mid-1990s. Since starting his career in 1971, as a programmer of applications in technical engineering, Hermann has built a deep and extensive understanding of the practicalities of using methodologies to produce successful software.

In this chapter, Hermann shares his experiences in CS's transition to service orientation.

Introduction

Like many large financial organizations, CS has grown significantly, from relatively small beginnings since the 1970s. This period has witnessed huge changes in business and technology, which continue unabated. Service orientation has emerged as an important part of our strategy for managing these changes. However, it is an ongoing journey that has required careful nurturing. It has not happened “overnight.” In this chapter, we examine some of the lessons that we have learned along the road to service orientation and at how CS is responding successfully to the continuous demand for change.

Historical IT background

In 1974, the IT system of Schweizer Kreditanstalt consisted of one IMS mainframe system. In 1975, for reasons of capacity, the system was split into two IMS mainframe systems to support regional areas. In 1980, a new application, Wertschriften System Asset Management (WS80), was introduced. Three IMS systems were necessary for reasons of independence and the growing demand. In 1985, greater demands for availability and capacity, as well as 7/23 hours operation, made a fourth IMS system necessary.

The idea of service orientation

The phrase, “Easy to do business with” has become a major driver for most companies. The idea of service orientation is to provide customer value by contracting others to do what a company has to do just to get by, and by focusing the company's own resources on what it does best. By subscribing to the commodity functionality provided by service providers who can perform it better, faster, and cheaper an organization can minimize its cost of market participation. This strategy releases energy for an organization to concentrate on its core competencies, thus bringing value to market through what the organization does best, thus making it easy to do business with. By taking a service-oriented approach, in an increasingly complex and competitive market, products can then be taken to market quickly and business processes conducted in agile response to change through multiple channels.

Such is the compelling attraction of service orientation. But “Wait a minute,” we hear top management say, “We might be serving the same customer, but we are a complex organization with multiple business units each with different targets, and supported by different technologies with different capabilities. We'll let the faster-moving, more profitable business units continue to do their own thing and just outsource the rest without worrying too much about the provider. That way we'll be service-oriented but we can't afford an integrated approach.”

Most organizations are not able to meet the challenges of service orientation with a “clean slate” – they do not change overnight. In part 2, we therefore provide an approach to business architecture (BA) that is based on evolution of best practices and in tune with the need to provide useful business process improvements quickly. We develop an example that continues through the remainder of the book and that illustrates integration of the BA with both SOA and SOM. The main focus is on the following aspects of the BA:

• business process models

• service policy

• domain and service models.

Chapter 4 provides foundation concepts for understanding services in the business context. In particular, we provide a service-oriented business process redesign pattern, and discuss how sourcing and usage strategy influences our approach to redesigning processes.

Chapter 5 focuses on the business process model. We provide a simple example of tactical first steps in moving toward service orientation based on incremental business process redesign given an existing legacy portfolio. We also consider the concept of service-oriented viewpoints in more detail, and show how to apply it to achieve a better business process redesign.

Chapter 6 discusses service policy, domain models, and service models. We address the longer-term, strategic view of identifying and designing services. In particular, our focus is on exploring how services can be used in different business contexts. We wrap things up by discussing the key aspect of surveying and cataloging services as corporate assets.

Cultural shifts

In this chapter we outline the BPM technologies and the associated gear shifts in business modeling that are required for effective service orientation. We first take a brief detour into the history of BPM technology before outlining the key elements of an effective BPM solution and previewing the main differences between a service-oriented approach and traditional approaches to business modeling. In this section, by way of prelude, we take a look at some of the cultural shifts involved.

It is important to restate some ground rules up front. We are focusing in this chapter on services as a way of enabling federated business processes (note for brevity we will refer to “business process” as “process” throughout, unless the context requires otherwise). Software services are services offered in software. Web services are but one manifestation – albeit a very compelling one – of software services. We shall therefore (as before) refer specifically to Web services, rather than software services, only where context demands it – as, for example, in the technology background to BPM.

Cultural shifts in IT

IT organizations continue to find themselves under increased pressure to find ways of doing more with less. In many cases, the development and integration efforts of the 1980s were designed to increase organizational efficiency by automating departmental functions. In the 1990s, the agenda moved toward using automation to help streamline internal processes and to removing the “white space” (Rummler and Brache, 1995) on the organization chart.

Introduction

We now turn toward scaling up the early efforts at business process redesign. In chapter 5, we sketched out some tactical scenarios that illustrated the use of existing assets to start to redesign existing processes. Web services were used in opportunistic fashion to leverage some of these assets through wrapping. The Web services were little more than single operations applied to very specific business requirements.

In this chapter, we move to the longer-term, strategic view to identifying and designing services that can meet the wider SOV7 requirements. This will involve considering the aspects of the BA shown in figure 6.1. We will move from left to right across this picture, starting with the service policy aspects, then considering domain models, and finally considering the two service models on the right-hand side. In particular, we focus on exploring how services can be used in different business contexts. These considerations in turn will cause the processes to be re-evaluated and developed so that they become more service-oriented.

A major part of this work concerns surveying and cataloging of corporate assets, of which services are a part. At the end of this chapter, we therefore also consider the place of the asset inventory in relation to this work.

Service policy

Service policy covers business goals and rules, the business–IT alignment table (BIAT), and sourcing and usage policy: the four items shown on the left of figure 6.1.

Foundations of SOA policy

Recall from chapter 4 that business goals and rules, the BIAT, and sourcing and usage policy are captured as part of the BA policy. Traditional approaches tend to divorce business concerns from IT. In contrast, as we shall see in this chapter, a service-oriented approach involves a very close relationship between business and IT concerns, between BA policy and SOA policy.

Another problem with traditional approaches is that they tend to focus on business functionality and information, and to neglect the area of policy. In contrast, a service-oriented approach emphasizes policy as much as functionality and information requirements. In this chapter, we introduce the policy aspects of SOA along with techniques for capturing and developing that policy in alignment with business needs.

SOA policy aspects

The policy aspect of SOA, shown in overall context in figure 8.1, falls into four parts:

• QoS

• Design

• Sourcing and usage

• Technology.

While design, sourcing and usage, and technology all represent critical areas of SOA policy, our view is that progress in each of these areas has been up to now much better than in the case of QoS. To a large extent we can evolve what we already have learned regarding policy in the areas of design, sourcing and usage, and technology. Indeed, very often organizations have no choice but to evolve in a manner that seeks to maximize return on existing assets. Business leaders understandably will have it no other way.

Preparing for service-oriented management

Some of the greatest challenges surrounding the industrial uptake of service orientation surround the abilities of the supporting technologies to scale up in the face of ever-increasing numbers of transactions, across widening geographical and commercial boundaries, and in a manner that can be trusted by all participants. While this puts the onus on effective SEM software, which we briefly discussed in chapters 2, a precondition of effective SEM software is the ability to understand and to measure what is actually to be managed. Unfortunately, the IT industry does not have a great track record when it comes to measuring software quality; “good enough” software has been the dominant trend for many companies for many years now in the face of shrinking budgets and development timescales. Software quality or nonfunctional requirements, to use the industry lingua franca, have always been something of a poor relation of functional requirements!

The challenges presented by software services, with their capability to cross traditional boundaries, including security zones, are an order of magnitude greater than with traditional software. The more pervasive software services become, the more critical is their role in the success or failure of the business. Effective SEM must expose and monitor executing applications so that problems can be detected and corrected as early as possible.

So things must change if service orientation is to take off successfully. The requirements for capacity, availability, and security of services must be clearly specified on the basis of measurable quality attributes.

Basic principles

We have emphasized that the SOA evolves over time in coordination with the business process redesign efforts. Now it is time to take a closer look at the concept of SOA itself. Unfortunately, SOA has become a much overused and abused term, subject to vague interpretations. It is important to clarify what exactly is involved in the idea of an SOA and how it differs from previous approaches to IT architecture. In this short chapter, we therefore provide an overview of the elements of SOA and outline the overall approach to evolving the SOA.

It is probably fair to say that the topic of SOA is as much about attitudes and cultures as it is about concepts. Apart from the shifts already discussed in the BA area, there are some other major shifts in mind-set required, that we discuss in this chapter.

While the SOA helps to plan strategies and specify services, the software has to work, and work extremely well in demanding conditions. Enter, therefore, SOM. In this chapter, we elaborate on the concept of SOM (introduced back in 2.3) and outline its relationship to the SOA. Later in this book, we shall build upon these introductory guidelines.

The elements of the SOA

What exactly, then, is the SOA? In this section, we provide the short answer in terms of a high-level bill of materials. There are two major aspects of the SOA: policy and structure.

The policy aspect of SOA falls into four parts:

• QoS

• Design

• Sourcing and usage

• […]

In the broader scheme of things our use of computer systems is still in its infancy. In just fifty years we have made amazing progress, to the point that computer systems are essential to the smooth running of home and business. Yet any systematic or casual measurement of user satisfaction with enterprise systems will inevitably return unsatisfactory ratings. The problems are legion but, with monotonous regularity, surveys report that users believe that total costs of ownership are too high and systems are insufficiently responsive to constantly changing circumstances.

The root problem is that insufficient attention has been paid to the fundamental structure of software systems. In other engineering disciplines such as automotive, construction, or electronics, there is unambiguous understanding between the user requirements and the delivered product, such that the various stakeholders can engage with the architect in a highly meaningful manner. For example, the architect and designer of the automobile, bridge or silicon chip has well-understood responsibilities to optimize the design and economics by using standard patterns and components which deliver a product that has a well-understood life cycle from production engineering through to retirement.

In contrast, the typical software system has no coherent structural and economic model that governs its life cycle. Rather, the approach to acquiring software systems is usually a mixture of blind faith and optimism.

Substitute business success for genius and this familiar aphorism applies with even greater force. When it comes to IP, the strategizing part is the 1% and the rest is implementation. But, after a seemingly sensible strategy has been put into place, corporate grandees may be tempted to pat themselves on the back and bow out, leaving the details to trusted vassals. It is not overstating matters to say this will probably doom the strategy – not only because 99% of the journey lies ahead, but because only through implementation can the strategy itself become refined and respond to ever-changing business circumstances.

Dialog is the current that drives this feedback loop. Central to effective decision-making in any enterprise, dialog means engagement rather than one group listening to its own voice. All key constituencies – upper management, marketing, and technical – must participate. In ways that make sense, of course: upper management need not get mired in the minutiae, nor should the hands-on IP administrators make all the decisions. How to tether the different players together and orchestrate their efforts into an integrated process is what implementation, and this chapter, are all about.