In this chapter, we discuss the relationship of open source to components and frameworks. We also explore how open source can be of benefit in framework development, and how and why to draw the line between the open-source world and the commercial software world in the right place for your particular project. Although Chapter 8 concentrates on understanding, selection, and use of components and frameworks, this chapter deals with their development and ongoing evolution, specifically in relation to the open-source community.

OVERVIEW OF OPEN SOURCE

First, let us examine what open source is all about. We examine what the term means, how the concept translates into reality, and its impact on the software industry.

What is Open Source?

Open source is a term that is widely used in today's software industry, but it is not always clear what it exactly means. Many people have a specific idea in mind when they hear the term, but misconceptions abound.

Open source seems to imply free of cost, and often this is the case (although not necessarily). Many believe it means complete freedom to do what they wish to do with the software: to copy it, redistribute it, and so forth. This is not necessarily true at all.

Many people also have a negative reaction to open-source software, believing that some aspects of it will be harmful to their business or organization. Interestingly enough, they are sometimes right – though more often wrong.

Over a number of years of working with frameworks, and being involved in their development and use, I have found that a few recurrent themes have come up. A number of issues arise again and again, and various solutions are tried. Sometimes the solutions work out, and apply to the same kind of problem when it comes up again in the future. Just like the design patterns and structure that evolve in a framework, these kinds of “best practices” that have evolved over time can save you considerable effort when applying frameworks to your own development projects.

In this chapter, we explore some of these practices and discuss how they came about, and how they can be applied to your own situation.

INITIAL ADOPTION

The first group of best practices that we discuss relates to the initial adoption stage of the use of frameworks: when you are just getting started there are some points to consider. This is the stage where a good number of mistakes are made, sometimes resulting in an otherwise promising framework being abandoned.

Take the Plunge

The first most important strategy for using frameworks and components in your development process is to make the decision to do so in the first place. We have explained in great detail, in the preceding chapters, how frameworks bring benefits to the development process, discussed what these benefits are, and examined concrete examples of frameworks.

GENERAL PRINCIPLES OF IDES IN FRAMEWORK/COMPONENT DEVELOPMENT

Integrated Development Environments (IDEs) are often a software developer's most intensively used tool, and this is often true of Web-application development. Until recently, IDEs had few special features for Web-application development, but this has changed. IDEs have become more flexible and easily configured, and their increased capabilities make them a better fit with frameworks. In this chapter we examine IDEs and how they relate to frameworks, with a few specific examples as well as some general observations and best practices.

IDEs and frameworks actually have much in common. They are tools that allow you to build your applications. The difference is that whereas an IDE provides tools for manipulating code, editing, searching, compiling, and so forth, a framework also provides services to be used by your code. A framework does not generally help you manipulate code (although some do generate code), it instead provides you with structure and services to build your application faster and easier.

Sometimes the line blurs a little between a framework and an IDE. Some IDEs provide libraries or APIs that are intended to be used in combination with your finished application at run time – this means they have at least some of the elements of a framework. Some frameworks provide graphical or command-line frontends to help you generate code, where the generated code is commonly a subclass of one of the abstract classes provided by the framework – in this sense, the framework provides some of the capabilities of an IDE.

INTRODUCTION

The best way to get a “feel” for different frameworks when choosing between them is to use them – to put them to work thereby creating an actual application. In actual practice, this is seldom done. No one has enough time to spend creating throw-away applications in several different frameworks. Once a developer begins to use a particular framework, and dedicates time to learn it well enough to do the example, it is easy to continue, and to take advantage of what has been learned so far. To switch tracks and use another framework is difficult, and things that have been learned about the first framework must often be unlearned to properly use another.

The comparison is still very helpful; however, seldom it is done. Creating a similar application in multiple frameworks, even a very simple one such as the one used here, allows a point-for-point comparison between frameworks. If one of the frameworks you are giving serious consideration to is not one of the ones we have chosen for a case study, you might find it helpful to implement the same sample application as a starting point.

In this Appendix, we use several detailed code examples, so that you can see what developing the same application with different frameworks is like. We do not, for lack of space, list every source code line here. You can download a zip file of the complete source code from http://www.cup.org/titles/catalogue.asp?isbn=0521520592.

WHY ARE COMPONENTS THE FUTURE OF WEB-APPLICATION DEVELOPMENT?

In this chapter we discuss the why of components and frameworks and the rationale behind their use. We also talk about the current state of Web-application development and where the industry perceives it is going to establish a firm foundation and justification for the use and development of components and frameworks.

We also look briefly at Java's suitability for component-based development and for the development of application frameworks, as well as for the specialized features of the extended Java platform and associated APIs that make them ideal to this task, including JavaBeans, Enterprise JavaBeans, and Reflection.

Where We are Today

The software industry is, for the most part, still creating much of its product in a “monolithic” fashion. The products may be more modular and configurable than they used to be, but most projects cannot be said to be truly component based. Even some projects being built with component-enabled technologies are not taking full advantage of the component model. It is quite possible to misuse component capabilities and as a result, to forfeit many of their benefits.

Many companies and organizations are becoming aware of the advantages and are getting their developers trained in the new technologies and the proper way to use them. It takes time for an organization to adopt such a significant change in their current practices.

Does technology encourage or inhibit business innovation? At first blush, the question seems naïve and the answer obvious: It is clear that information technology is an enormous spur to innovation. New technological capabilities enable companies to perform activities of which they had previously only dreamed. Modern data management facilities, for example, allow organizations to conduct customer analyses that they always had desired but previously had been unable to do and thereby develop customized offerings and marketing messages. In some cases, new technology even allows companies to solve problems of which they had previously been unaware. A classic example is the invention of the xerographic copier. Prior to its arrival in the marketplace, people did not have an expressed need for such a device. They used carbon paper and similar technologies to make extra copies of a document while it was being produced and resigned themselves to living in a world where one could not make copies thereafter. Indeed, early market research studies showed no demand for a convenience copier. Only later, as people started to appreciate the capability that the new technology offered them did they recognize the opportunity that it represented.

At times, however, technology can have the opposite effect and inhibit business innovation. In particular, investments in expensive technology platforms represent a “sunk cost” for most organizations, a cost that they are reluctant to reincur.

INTRODUCTION

Much has been accomplished by designing information systems architecture as independent tiers that interoperate only with those that are directly adjacent. In this way, logic is boxed into a logical hierarchy that makes solutions more scalable and easier to understand, maintain, and change.

This thinking started with the client/server model. This model was used successfully at first in two different environments and for two different purposes:

• as a way to separate the database services from an application that uses them (in this case, the client is the application and the server is the database server),

• as a way to separate the presentation services on a workstation from an application that uses them (in this case the client is the application and the server is the presentation server).

As more modern enterprise applications evolved, the need for separating both the presentation layer and the data layer from the “business-logic” layer became self-evident, giving way to the so-called three-tier client/server model. The three-tier architecture has had many different implementations and has gotten more and more sophisticated.

Initially, the layers were simply the data layer, which consisted fundamentally of the database server; the presentation layer, which was basically a presentation server; and the application layer in the middle, which consisted of everything else.

The application layer still implemented many generic reusable services that could be distilled from it. As these services started being abstracted from the application logic, the concept of application servers as “business logic containers” became more popular.

INTRODUCTION

So far, we have spent a great deal of time in covering the reference architecture for business services orchestration, and defining the necessary methodology and the underlying technologies that make it possible to orchestrate business services. The actual development process for coming up with orchestration sequences is covered in the methodology section. Additionally, methods and processes, such as the Unified Process from Rational, are also a great source for going through the modeling process. This chapter devotes itself to the key concepts that need to be modeled. We talk about the necessary components of a BSO, which need to be visually rendered through a BSO notation and need to be captured by a BSO specification language (BSOL).

BSOL captures the protocol that the collaborating systems need to follow in order to interact with each other in a meaningful business manner. In a typical BSO scenario, there are many business services. These services have their public interfaces, which may have been captured by a specification language such as Web Service Description Language (WSDL). The orchestration of these services is all about how these services interact with each other in a state-full manner, typically in long-running business transactions. A buyer may send a purchase order to a seller. The seller may take several days to fulfill the order and then send an invoice back to the buyer. Although the buyer and seller services have their interfaces, the protocol defines the order in which those interface operations can be invoked.

INTRODUCTION

Applications in our context are defined as software that is developed by programmers to solve specific business problems. These applications generally implement business logic which is either not available in off-the-shelf applications, or some special constraints such as performance or security, which are otherwise not met by the COTS applications have to be considered.

Figure 3.1 describes the reference model for BSO. Our discussion in this chapter relates to developing software that resides in the Business Services tier and the tiers above it. Most orchestration products provide an orchestration engine, a sort of maestro, which reads the BSO definition and then manages the flow of activities. In some implementations, the orchestration engine as well as business logic code may execute inside a container environment. We discuss application and data services in this chapter as seen not only by the application containers but also by the orchestration container and the orchestration engine.

Products in this functional category, discussed in Section 5.2, relate to the Web application servers layer of the reference architecture presented in Chapter 3, Figure 3.1. Programming languages discussed in Section 5.3 are important for all layers of the reference architecture. Technologies discussed in Section 5.4 are again important for all layers in the reference architecture but relate more to the packaging protocols layer of the stack.

APPLICATION DEVELOPMENT PLATFORMS

An application development platform is a comprehensive set of services to develop, host, and manage applications. Any distributed application should be built using a robust application platform.

INTRODUCTION

No groundbreaking work on business services orchestration (BSO) would be complete without establishing a robust underlying methodology. We have developed a specific methodology for orchestrating and improving business services, which, when used with a specific orchestration suite, results in the greatest return on investment (ROI) for clients.

This comprehensive approach is the culmination of many implementations of orchestrations accomplished by developing and using a BSO toolset. It focuses on the fundamentals of services analysis and design and process modeling as a form of orchestration implementation. It includes specific techniques and tools that we have developed by modifying the best approaches to services discovery and design.

Among the initial results of applying this methodology, we can mention the renewed role of the chief information officer (CIO). CIOs who master this approach and make it an integral part of their mode of operation cease to be seen as the nay-sayers in the company. They no longer adopt the role of gatekeepers and fire extinguishers. Instead, they become agents of change. They become the hinge between the company's strategy and the implementation and continuous improvement of the services that execute it. They no longer act as a service center for the company's users; they become the catalysts for improved services to the company's customers. There are companies where the CIO does not have the business wherewithal or interest to take on this catalyst role. In those companies, the LOB managers, the COO, or even the CEO should take ownership of the initiative.

INTRODUCTION

The first part of this introduction is for the Kirks, Spocks, McCoys, and Scottys. As it gets more technical, the Kirks may want to skip directly to Section 1.3, from which point the content is more business oriented.

To Orchestrate is to organize the harmony and tempo of various instruments into a single impressive musical delivery, such as in Strauss's Blue Danube waltz. If the result is anything but impressive, the orchestration is not worthy of such a name. An Orchestrator differs from an Architect in that the latter designs something static, such as a house, a bridge, or a landscape, that is, something that doesn't vary in time. The Architect designs only the deliverable, not the process used to deliver it. An Orchestrator designs a delivery of music rendered in time, in the harmony and tempo needed to achieve a desired effect.

Orchestration is nothing but a modern metaphor that describes a well-known, but not very well understood, discipline: the automation of business process management. Traditionally, business processes were managed by people. Managers drove the steps that employees – with or without instruments or tools – needed to complete or fulfill a business process. This is analogous to an orchestration being managed by a maestro by keeping tempo, cueing in the different players, managing the intensity of what is being played, and conveying a style to the performance.

To start, we would like to set the stage. Please read slowly, and use your imagination. Remember the scene in 2001: A Space Odyssey where the Blue Danube waltz is sounding. The space shuttle is rotating on its axis trying to match the rotation of the space station. Perfect synchronization. Harmony. Striking beauty. A fantastic Orchestration.

For these first paragraphs only, we will use Star Trek as an allegory while the finely orchestrated waltz of 2001: A Space Odyssey sounds in the background. We will say that the Federation (as in Star Trek) is the modern enterprise, the president of the Federation its CEO. The USS Enterprise is one of the many starships that are capable of performing a mission according to a predefined plan (implementing an Orchestration): an orchestration vehicle. Captain Kirk is the mission owner (Orchestration owner) for this orchestration vehicle. He is ultimately responsible for the different possible missions (deliverables) and maintains the captain's log. He is also responsible for overseeing the execution of the process (Orchestration) that fulfills the mission. He is not an expert in managing the engine; that's Scotty. He's not an expert in connectivity; that's Ohura. He's not an expert in dealing with the crew's ailments; that's Dr. McCoy.

The USS Enterprise needs to be able to use its own internal services (engine, molecular transporter, computer, view screen, navigation equipment, communication equipment, energy sources, crew services, etc.) at different points in the space-time continuum to complete the mission.

INTRODUCTION

Service aggregation is, simply put, service integration. Aggregated services are generally produced by consuming the contracts of multiple services to produce one single unified contract. They represent either functional composition where higher-level business functions are produced by consuming contracts of multiple services at the back end or they simply represent a larger contract that is the sum of its parts. These aggregated services present a contract of their own to their clients and are in turn consumers of the services they are aggregating.

Multiple back-end services are summed up in the contract of an aggregated service to either extend the reach of those back-end services or present a more convenient point of access for the clients. In either case, the motivation is based on technology and convenience. As an example, an enterprise may have various services, all related to one particular business area – say inventory – for its intranet. However, this enterprise decides to outsource its inventory management so that its inventory is visible to its suppliers and they can automatically replenish it as needed. A solution would be to develop a Web service that aggregates all the internal inventory control services and presents an aggregated contract to its business partner to help with the management.

Functional aggregation is more driven by the business need to develop increasingly complex functions and by the need for applications to evolve functionally.

INTRODUCTION

As with everything else, we have a very holistic view of metadata and discovery features, looking at the entire reference model described in Chapter 3 and shown in Figure 3.1. In this chapter, we describe all different kinds of metadata that are relevant for a complete orchestration environment. We discuss the role of service discovery facilities and describe what is expected from those facilities.

The role of metadata and service discovery is generally well understood in a service-oriented architecture. Figure 7.1 illustrates the typical publish–discover–bind cycle. A new business service is developed. Its developers publish the service contract in the business registry. The service consumers discover the service from the registry, bind to it, and use the service. In this typical service-oriented architecture, the service contract consists primarily of the service description. This service-level information is generally categorized using some taxonomy scheme. A service consumer either searches the service by name or looks for a service that implements some particular specification.

Although these roles of metadata and service discovery are still true for a complete BSO environment, the metadata for the BSO environment are significantly more complex than what was described above, and service discovery requires some additional capabilities as well.

Metadata can actually be split into four broad categories:

1. Descriptive metadata. These metadata include service contracts, orchestration definitions, and messages data types.

2. Rules. An important part of metadata is various kind of data validation and data transformation rules that have been defined for successfully execute orchestrations.

3. […]

INTRODUCTION

An objective of business services orchestration (BSO) is to remove or minimize human intervention that is not justified by business considerations. This is driven by the motivation to reduce swivel-chair operation, thereby reducing latency between activities, and to minimize errors. However, human intervention is sometimes extremely necessary. For example, let us consider a scenario in which a procurement request is entered into the system. Because of legal ramifications and internal policies, a human has to approve and digitally sign the request. Because we view everything in the service-oriented architecture as a first-class service, we view human participation as a service that is provided by people, similar to the service provided by a software application.

BSO sees human participation in a process as a service. This view of human services is germane to true BSO. It's not capricious. The reason for this level of abstraction is that the ultimate goal of any BSO implementation is to replace routine, repeatable services from people with services from applications that automate them. When human participation in a process is treated by the process as a service invocation, it becomes transparent to the process whether the service is from a human or from an application. In fact, it is always from a human through an application anyway.

The standard application component of a BSO suite that is used to invoke services from people is the work portal.

INTRODUCTION

At this point we expect the reader to be comfortable and familiar with what we call business services, the different classifications, and what we envision to be the future of technology.

In this chapter, we explain what orchestration is by explaining the process of orchestration and then contrasting the resulting paradigm against the integration, automation, collaboration, BPR, and Web services paradigms. Then, we talk about the impact of orchestration on information technology (IT) disciplines and the way we do business.

Without any doubt, orchestration is something that, as of today, only humans can do. It is a creative activity. Furthermore, in today's enterprise, services are being orchestrated on-the-fly. Who is doing this? People are. People request services from other people and/or systems to be able to perform a service for someone else. Among the natural orchestrations that people perform, there are those that are systemic and repetitious, and there are those that are “one-offs.” These systemic, routine orchestrations can be performed by an orchestration engine, relieving people of routine, repetitious tasks. From this perspective, orchestrations consist of capturing the rules and sequences of how and when a person invokes services to render a new one. In other words, orchestration consists of creating an executable process model that implements a new business service by harmonizing preexisting business services and managing their tempo.