Why this book?

We live in a world of change. During my thirty-plus years in the software industry, I have witnessed some remarkable changes. One constant, however, seems to me to be that successful companies cope with change by nurturing their people, learning fast, building on what they have learned, and discarding things that no longer work. Surely, that's not rocket science! If that's the case then, “Why,” I ask myself, “write a book about it?” It seems appropriate to begin by offering the reader some kind of answer!

Even moderately large companies can no longer afford to “just get by” as they seek to cope with the challenges of a world of unforgiving change. Business is increasingly moving toward a marketplace model. This model is in sharp contrast to the traditional view of an organization as a production line. In this world, organizations collaborate together, consuming and offering services to maximize efficiency, better serve customers, and achieve long-term advantage.

This is the world of service orientation, in which a hasty decision to outsource an activity may well be a decision that the firm repents at leisure. Equally, focusing the company's own energies on the wrong kinds of thing could be a disaster. The power of technology, from the Internet to grid computing and Web services, is the irresistible force that is fueling this change. Business is increasingly less supported by software and more enabled by it. This means that simply automating business activities in software is no longer enough.

This first part of the book sets the context for the remainder of the book by explaining the principles of service orientation, as a business phenomenon enabled by developments in technology.

Chapter 1 provides a conceptual overview, defines the core terminology and introduces the pivotal idea of service-oriented architecture (SOA). Service orientation presents some massive cultural and technical challenges that cross three areas that have traditionally worked largely in isolation from one another: business process improvement, application development, and software operations. We provide a simple example of how the idea of a service can provide a unifying thread for drawing together these areas, along with some guidance for selling this approach to business management.

The remainder of part 1 maps out the foundation technologies required for practical application. Most of these technologies are developing out of previous well-established generations of technology. At the same time, they are based on standards that are complex as well as changing. Rather than attempting the impossible task of reviewing and documenting all of these technologies and standards we provide an essentially abstract primer designed for maximum resilience to the pace of change; a list of useful evolving Internet information sources is provided at the back of the book. For example, our emphasis is on the general idea of a software service, rather than specifically on a Web service, unless of course the context demands it.

SOA has received its fair share of interpretations. It is a term that is commonly bandied about without clear definition. In fairness, the topic of SOA is probably as much about cultural change as it is about concepts and techniques. However, SOA is integral to the approach described in this book and therefore it is important to define our terms very clearly.

Part 3 of this book lays out the two major aspects of the SOA: policy and structure. We explain how to achieve an integrated approach to SOA that is driven top-down by the BA, and equally that is supported bottom-up by SOM.

Chapter 7 sets the context for the remainder of part 3 by providing an overview of SOA concepts and shows how the essentially static view of the SOA interconnects with the dynamic view of run-time services provided by the SOM. This paves the way for our discussion of SOM in part 4. We also provide an overview of the required changes in mind-set.

Chapter 8 deals with the policy aspect of SOA and provides techniques for the business–IT alignment of policy. We provide an overview of the four main areas of SOA policy – QoS, design, sourcing and usage, and technology – and preview the four key types of QoS requirement – agility, capacity, availability, and security.

Chapter 9 centers on agility, and provides a compendium of service design techniques for structuring the SOA and ensuring business–IT alignment.

A cohesive approach

The IT industry is full of niche comfort zones. One of the unfortunate consequences of the niche mentality is that the execution management of software is treated as an entirely separate subject from topics such as analysis, design, and software development. While separation of concerns is necessary for specialization and discipline, it becomes unhealthy when important interconnections and relationships between domains of interest are ignored – when separation leads to divorce!

In our present context, this trend surfaces in the separation of SOA from both overall operational management of IT services (in a general sense) and the specific execution management of software services. The consequence of divorcing these subjects is extra painful in the case of services. In particular, if services are to adapt in an on-demand fashion, execution management of the services must ensure that SLAs are satisfied, in compliance with SOA policy, with respect to ever-changing run-time conditions. As we argued in chapter 2, these new challenges call for gear shifts in our approaches to service management, toward SOM: a discipline aimed at ensuring that our architectures (BA and SOA) are not just “theory,” but actually reflected in real production running software services.

SOM is dependent upon good specification of the services that it seeks to regulate. To underline a constant theme of this book: “You can't manage what you don't measure,” but more than that “You can't measure what you don't specify.

Agility

Software applications are seldom built for the sort of general purpose uses that a computer is suitable for. Despite our ingenuity in building computer chips that can run everything from aircraft to video games, the software itself is good for only one kind of activity or another. The result is that all too often software that is anything but “soft” – it is very brittle and difficult to change.

In chapter8, we emphasized that an SOA addresses this long-standing problem by treating software quality – and in particular agility – as an integral part of software architecture. It is important to note that this does not mean formalizing the development process, as in the “engineering” approach to software quality, exemplified by the CMMI, the established approach to formalizing the processes of creating and managing software.

The fact is that when we build software to meet specific, predefined requirements, it almost always falls short of meeting the true requirements for that software, primarily because the actual requirements for software are typically in a state of constant flux. Software must be agile enough to meet as yet undefined requirements at some point in the future.

In this chapter, we therefore focus on how to design agile services. This takes us to the heart of SOA: modeling techniques that help get the right set of services.

Sam Higgins is now with Forrester Research, Inc. as a senior analyst. Formerly he was a director of Encode Services, an Australian information and communications technology consulting firm, and managed application architecture direction for the Innovation and Planning unit of Queensland Transport's Information Services Branch. He has over twelve years' experience in both traditional and model-driven environments, and is a popular international speaker. Sam was a driving force behind Queensland Transport's adoption of the Java 2 Enterprise Edition (J2EE) platform and subsequent Web services implementations.

Paul McRae is the application architect for the Innovation and Planning unit of Queensland Transport's Information Services Branch. He has almost twenty years' experience in the IT industry, was a key player in the introduction of an SOA within Queensland Transport, and was instrumental in developing Queensland Transport's CBD initiatives, J2EE applications, enterprise architecture, and Web services.

In this chapter Sam and Paul share their experiences in Queensland Transport's transition to service orientation.

Background

Queensland is Australia's north-eastern state, covering nearly 1.7 million km2, and home to approximately 3.6 million people. Known as the “Smart State,” Queensland is a center of significant leading-edge technology developments. Queensland Transport is a department within the State Government of Queensland.

Queensland Transport has a budget of approximately A$1.3 billion and over 4,700 staff throughout the state, and is responsible specifically for the regulation of land, sea, and aviation transport in Queensland.

Hermann Schlamann is a senior architect in the architecture group of Credit Suisse (CS), having worked as a consultant in the banking industry in Switzerland since the mid-1990s. Since starting his career in 1971, as a programmer of applications in technical engineering, Hermann has built a deep and extensive understanding of the practicalities of using methodologies to produce successful software.

In this chapter, Hermann shares his experiences in CS's transition to service orientation.

Introduction

Like many large financial organizations, CS has grown significantly, from relatively small beginnings since the 1970s. This period has witnessed huge changes in business and technology, which continue unabated. Service orientation has emerged as an important part of our strategy for managing these changes. However, it is an ongoing journey that has required careful nurturing. It has not happened “overnight.” In this chapter, we examine some of the lessons that we have learned along the road to service orientation and at how CS is responding successfully to the continuous demand for change.

Historical IT background

In 1974, the IT system of Schweizer Kreditanstalt consisted of one IMS mainframe system. In 1975, for reasons of capacity, the system was split into two IMS mainframe systems to support regional areas. In 1980, a new application, Wertschriften System Asset Management (WS80), was introduced. Three IMS systems were necessary for reasons of independence and the growing demand. In 1985, greater demands for availability and capacity, as well as 7/23 hours operation, made a fourth IMS system necessary.

The idea of service orientation

The phrase, “Easy to do business with” has become a major driver for most companies. The idea of service orientation is to provide customer value by contracting others to do what a company has to do just to get by, and by focusing the company's own resources on what it does best. By subscribing to the commodity functionality provided by service providers who can perform it better, faster, and cheaper an organization can minimize its cost of market participation. This strategy releases energy for an organization to concentrate on its core competencies, thus bringing value to market through what the organization does best, thus making it easy to do business with. By taking a service-oriented approach, in an increasingly complex and competitive market, products can then be taken to market quickly and business processes conducted in agile response to change through multiple channels.

Such is the compelling attraction of service orientation. But “Wait a minute,” we hear top management say, “We might be serving the same customer, but we are a complex organization with multiple business units each with different targets, and supported by different technologies with different capabilities. We'll let the faster-moving, more profitable business units continue to do their own thing and just outsource the rest without worrying too much about the provider. That way we'll be service-oriented but we can't afford an integrated approach.”

Most organizations are not able to meet the challenges of service orientation with a “clean slate” – they do not change overnight. In part 2, we therefore provide an approach to business architecture (BA) that is based on evolution of best practices and in tune with the need to provide useful business process improvements quickly. We develop an example that continues through the remainder of the book and that illustrates integration of the BA with both SOA and SOM. The main focus is on the following aspects of the BA:

• business process models

• service policy

• domain and service models.

Chapter 4 provides foundation concepts for understanding services in the business context. In particular, we provide a service-oriented business process redesign pattern, and discuss how sourcing and usage strategy influences our approach to redesigning processes.

Chapter 5 focuses on the business process model. We provide a simple example of tactical first steps in moving toward service orientation based on incremental business process redesign given an existing legacy portfolio. We also consider the concept of service-oriented viewpoints in more detail, and show how to apply it to achieve a better business process redesign.

Chapter 6 discusses service policy, domain models, and service models. We address the longer-term, strategic view of identifying and designing services. In particular, our focus is on exploring how services can be used in different business contexts. We wrap things up by discussing the key aspect of surveying and cataloging services as corporate assets.

Cultural shifts

In this chapter we outline the BPM technologies and the associated gear shifts in business modeling that are required for effective service orientation. We first take a brief detour into the history of BPM technology before outlining the key elements of an effective BPM solution and previewing the main differences between a service-oriented approach and traditional approaches to business modeling. In this section, by way of prelude, we take a look at some of the cultural shifts involved.

It is important to restate some ground rules up front. We are focusing in this chapter on services as a way of enabling federated business processes (note for brevity we will refer to “business process” as “process” throughout, unless the context requires otherwise). Software services are services offered in software. Web services are but one manifestation – albeit a very compelling one – of software services. We shall therefore (as before) refer specifically to Web services, rather than software services, only where context demands it – as, for example, in the technology background to BPM.

Cultural shifts in IT

IT organizations continue to find themselves under increased pressure to find ways of doing more with less. In many cases, the development and integration efforts of the 1980s were designed to increase organizational efficiency by automating departmental functions. In the 1990s, the agenda moved toward using automation to help streamline internal processes and to removing the “white space” (Rummler and Brache, 1995) on the organization chart.

Introduction

We now turn toward scaling up the early efforts at business process redesign. In chapter 5, we sketched out some tactical scenarios that illustrated the use of existing assets to start to redesign existing processes. Web services were used in opportunistic fashion to leverage some of these assets through wrapping. The Web services were little more than single operations applied to very specific business requirements.

In this chapter, we move to the longer-term, strategic view to identifying and designing services that can meet the wider SOV7 requirements. This will involve considering the aspects of the BA shown in figure 6.1. We will move from left to right across this picture, starting with the service policy aspects, then considering domain models, and finally considering the two service models on the right-hand side. In particular, we focus on exploring how services can be used in different business contexts. These considerations in turn will cause the processes to be re-evaluated and developed so that they become more service-oriented.

A major part of this work concerns surveying and cataloging of corporate assets, of which services are a part. At the end of this chapter, we therefore also consider the place of the asset inventory in relation to this work.

Service policy

Service policy covers business goals and rules, the business–IT alignment table (BIAT), and sourcing and usage policy: the four items shown on the left of figure 6.1.

Foundations of SOA policy

Recall from chapter 4 that business goals and rules, the BIAT, and sourcing and usage policy are captured as part of the BA policy. Traditional approaches tend to divorce business concerns from IT. In contrast, as we shall see in this chapter, a service-oriented approach involves a very close relationship between business and IT concerns, between BA policy and SOA policy.

Another problem with traditional approaches is that they tend to focus on business functionality and information, and to neglect the area of policy. In contrast, a service-oriented approach emphasizes policy as much as functionality and information requirements. In this chapter, we introduce the policy aspects of SOA along with techniques for capturing and developing that policy in alignment with business needs.

SOA policy aspects

The policy aspect of SOA, shown in overall context in figure 8.1, falls into four parts:

• QoS

• Design

• Sourcing and usage

• Technology.

While design, sourcing and usage, and technology all represent critical areas of SOA policy, our view is that progress in each of these areas has been up to now much better than in the case of QoS. To a large extent we can evolve what we already have learned regarding policy in the areas of design, sourcing and usage, and technology. Indeed, very often organizations have no choice but to evolve in a manner that seeks to maximize return on existing assets. Business leaders understandably will have it no other way.

Preparing for service-oriented management

Some of the greatest challenges surrounding the industrial uptake of service orientation surround the abilities of the supporting technologies to scale up in the face of ever-increasing numbers of transactions, across widening geographical and commercial boundaries, and in a manner that can be trusted by all participants. While this puts the onus on effective SEM software, which we briefly discussed in chapters 2, a precondition of effective SEM software is the ability to understand and to measure what is actually to be managed. Unfortunately, the IT industry does not have a great track record when it comes to measuring software quality; “good enough” software has been the dominant trend for many companies for many years now in the face of shrinking budgets and development timescales. Software quality or nonfunctional requirements, to use the industry lingua franca, have always been something of a poor relation of functional requirements!

The challenges presented by software services, with their capability to cross traditional boundaries, including security zones, are an order of magnitude greater than with traditional software. The more pervasive software services become, the more critical is their role in the success or failure of the business. Effective SEM must expose and monitor executing applications so that problems can be detected and corrected as early as possible.

So things must change if service orientation is to take off successfully. The requirements for capacity, availability, and security of services must be clearly specified on the basis of measurable quality attributes.

Basic principles

We have emphasized that the SOA evolves over time in coordination with the business process redesign efforts. Now it is time to take a closer look at the concept of SOA itself. Unfortunately, SOA has become a much overused and abused term, subject to vague interpretations. It is important to clarify what exactly is involved in the idea of an SOA and how it differs from previous approaches to IT architecture. In this short chapter, we therefore provide an overview of the elements of SOA and outline the overall approach to evolving the SOA.

It is probably fair to say that the topic of SOA is as much about attitudes and cultures as it is about concepts. Apart from the shifts already discussed in the BA area, there are some other major shifts in mind-set required, that we discuss in this chapter.

While the SOA helps to plan strategies and specify services, the software has to work, and work extremely well in demanding conditions. Enter, therefore, SOM. In this chapter, we elaborate on the concept of SOM (introduced back in 2.3) and outline its relationship to the SOA. Later in this book, we shall build upon these introductory guidelines.

The elements of the SOA

What exactly, then, is the SOA? In this section, we provide the short answer in terms of a high-level bill of materials. There are two major aspects of the SOA: policy and structure.

The policy aspect of SOA falls into four parts:

• QoS

• Design

• Sourcing and usage

• […]