To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Criticism is the application of a deep understanding of a topic or field to the newest creations of that field. Criticism can also place past works of art or creation into a current context by challenging icons, rediscovering lost achievements or redirecting the viewer, reader or participant to new modes of understanding. Individuals who make their living in the field are typically known as critics; those who do it part- time are usually called reviewers.
Why criticism?
Because all fields of human endeavor, in order to advance, need critical judgment to recognize genius, determine quality and otherwise set standards. Criticism should be subordinate to the works themselves, yet should hold equal weight in terms of value to society.
Types of criticism— Criticism can be distinguished, not just by subject matter, but also by venue. Thus, some are created under tight deadlines while others can take years to prepare. By the same token, the majority of reviews deal with a single book, musical or dramatic performance, television show, movie, or recording. But there is also a genre of criticism that looks at the entire oeuvre of an artist, placing him or her in the larger context of an entire era or creative school.
Here is a partial list of types of criticism:
Newspaper reviews— These have the tightest deadlines of any form of criticism. And they run the gamut from books to opera, plays and musical, musical performances, television and movies, audio recordings, video games, and so forth. It is the nature of newspaper publishing— the need to make the next day's edition— that creates the time pressure. Writing newspaper reviews typically involves attending an afternoon or evening performance then returning to the newsroom (or remotely filing via email) to write the story in the hours (or minutes) before the morning edition “goes to bed” (that is, sent for printing). At best— perhaps for a weekend book review section—reviewers may have a few days to prepare their copy. Though there has been some great newspaper reviewing over the last two centuries, it is definitely work in which erudition and strong opinions take precedent over elegant writing. Newspapers that serve large metropolitan readerships typically have full- time staffers as reviewers (though with the decline of newspapers, that is changing), while smaller- circulation papers usually hire freelancers.
Until now, this textbook has been about professional writing careers. We close with a look at the life of a professional writer.
The fundamental question
How should a writer live his or her life and conduct a career in order to achieve the greatest contribution, compensation, satisfaction and happiness? At the beginning of your working life this may not seem like a big deal: you're too busy with just trying to start your career, not worrying about how to finish it. But trust the author: there will come a time when you ask yourself whether it was all worth it, and what will be your legacy. The sooner you start asking yourself those questions, the better the odds that you will find (or more accurately, create) satisfying answers.
The following is the career wisdom of the author, acquired over the course of forty years as a professional writer. It is incomplete and, in some cases, perhaps even wrong— or at least a poor fit for your life. So, take it as you will, and keep searching for answers from every other professional writer you meet.
Recognize that you will change
Appreciate from the start that the person you are at 20 is not the person you will be at 40 or 60 or 80. Not only will your skills change, but also your interests and your dreams. Don't get stuck trying to live out fantasies that are no longer valid to your life— that is, just because at 20 you wanted to be a novelist, and told everybody so, doesn't mean at 50 you are still obliged to write that novel. No one cares. If something else inspires you instead, go after it— don't linger on the past.
Recognize that the world's perception of you changes as well
As you conduct your career, you will gain a reputation. Find out what that reputation is and don't delude yourself that you are seen as someone you are not. If your reputation is the one you want, then figure out how to cultivate and grow it. If it is not the one you want, then develop a strategy to change that reputation. And don't just sit where you are, trying to maintain the status quo, because your reputation will deteriorate if you don't tend to it.
In this chapter we look at two types of writing that appear in very different ways, but structurally, are nearly identical. Both are dialog- driven and both involve actors and actresses who perform the author's words and instructions. Where they differ has less to do with the writing and more with venue, modes of transmission, timeliness (one is “live,” the other not) and cost.
Screenwriting
What is screenwriting?
Screenwriting is designed to be viewed or heard, typically in a reading or performance that involves someone besides the author, and delivered by mechanical/ electrical/ digital means. Screenwriters produce screenplays, which are typically converted into working scripts, which are used to make motion pictures, Webcasts and television shows. Screenwriting can take the form of a monologue or dialog, and may or may not involve instructions to other creative personnel (instructions for camera shots, actor actions, etc. are usually reserved for the “shooting script.”
Screenplays share long- established formats that need to be followed by the screenwriter if she or he is going to have a chance at making a sale. You need to learn that format and be fluent in it. The world will not bend its century- old system to you; you must adapt to it.
Types of screenwriting
Though term “screenwriting” is typically associated with television or film, it actually encompasses just about everything creative you write for the electronic or digital media— even if it is just a few lines to announce the next show on the schedule. Here's some examples:
News stories— If you are writing for television or radio, the line between screenplay and script blurs. You are essentially writing copy, often under a tight deadline, to be read on- air. In this setting, you are often writing words to be either memorized and spoken, or to be put into a teleprompter to be read on air. Formatting is straightforward— it is rarely more than the news copy with the name of the reader(s) appended in a narrow margin, large font layout.
The authoring, co- authoring or compiling and introducing of a fulllength (40K to 300K word count) hardcover, softcover or electronic nonfiction book. Such a book typically takes 12– 15 months to complete, of which the actual writing is about 9 months. Time to publication after completion of the writing can range from days (e- books) to more than a year (traditional hardcover publishers). For the purposes of this book, we distinguish “author,” which we will use for nonfiction books only, and “novelist,” a term used exclusively for fiction.
Why write nonfiction books?
Because some subjects are of sufficient importance and complexity that they can only be treated at book length. Also, putting one's work between covers confers on it a value and gravity impossible in any other publication format. Finally, more than any magazine or newspaper article, books are substantial physical objects that can endure for years, even generations.
Types of books
The range of subjects for nonfiction books is as wide as the range of human interests. In other words, it is almost infinite. Here is a partial list of broad topics. Under each topic you can probably create a list of subtopics just as long. The point is that, especially in the modern global marketplace, there are probably enough potential readers in each of thousands of subgenres of nonfiction to make it commercially viable for you to write a book on that subject. And if you are willing to write out of love (or for reputation building) instead of money, the potential topics are probably ten times that.
The book-writing process
Writing a book is a major effort, one that can take thousands of hours and months out of your life. Too many people become obsessed with an idea for a book, sit down and just start writing from page one, devoting many hours per day for the first couple of weeks. This is a recipe for disaster— which is why only a small fraction of books started are ever finished. Most often, these writers simply burn themselves out. Or other responsibilities call, and they abandon the effort. Or they discover they don't have enough material for a book- length work.
This chapter looks at the many kinds of occasional writing that you, as a professional writer, may be asked to do. It is a varied list, but the writing remains the same. Economical writing, strong phrasing, action verbs, factual accuracy, careful editing; good writing is still good writing, even if it is for a neighborhood brochure.
Some of this writing will be for nonprofits, others for profit- making clients, and still others for small businesses with very limited budgets. Again, as a professional, you should never write for free— the exception being for a nonprofit organization. But with very small businesses you may want to modify your fee to match their budget— balanced by whatever goodwill you may obtain (for example, if you are asked by your favorite coffee joint to help it write a neighborhood flyer, you may want to take your payment out in free drinks).
It is also not recommended that you charge an hourly rate. People can look at a work of art— and because they can't draw themselves, it seems valuable. But everybody can write— though most can't do it very well— and so when they see your completed work they have a tendency to believe they could have done it by themselves, that it wasn't that difficult and as such they undervalue your work. Also, different writers work at different speeds, and many bad writers work much more slowly than good writers. So why charge by the hour? And why try to negotiate a fee after delivery?
Instead, charge by the project and agree upon the price up- front. If possible, also get paid half up front as well. Then, if your client decides that his teenaged daughter could have written that copy as well as you and refuses to pay for the finished writing, you at least have the equivalent of a kill fee in your pocket.
As for working with nonprofits, good for you for being a good neighbor and citizen. That said, nonprofits are in the business of getting good people with special skills to donate as much of their time as possible— and one of the skills they need most is writing.
“Blog” is a neologism created from the phrase “web log.” Strictly speaking, a blog is a web site on which an individual or group of users record opinions, information, interesting links, solicited comments and, more rarely, conduct traditional field reporting, on a regular basis. There are an estimated 200 million bloggers currently active in the world. The universe of bloggers is usually called the Blogosphere. While it is possible to earn a living as a blogger, the number of those who do so is extremely small, as the competition for readers is high, and the business model itself not particularly lucrative.
Why blog?
Blogging sits at the nexus of the natural human desire to share our thoughts and beliefs with others, with the hope that our words will live forever, and with the speed and infinite scalability of modern digital technology. Most bloggers get into the profession because of its very lowcost ease of entry (most blogging software is either cheap or free), and they stay in despite the limited financial return because they enjoy both the platform for their writing and opinions and the interaction with loyal and equally opinionated readers.
A brief history of blogs
Blogs began as extended diaries entries (hence “log”) and essays on the pioneering sites of the Internet, such as the Well, Compuserve, BBS sites, and so forth. Successful diarists— usually the most colorful and opinionated writers and experts in tech- related fields— found they were gaining loyal, even obsessive, followers and continued because they enjoyed the response.
Archival data suggests that the first real blogs appeared in 1994– 1995 at Swarthmore College by a student named Justin Hall— and by other pioneers such as science- fiction writer and essayist Jerry Pournelle, who moved his Byte magazine column “Chaos Manor” (founded in 1980) to the Web.
The transformative event in the history of blogging was the September 11, 2001 terrorist attack and the perceived failure of traditional media to keep up with the pace of news. The events of 9/ 11 made the reputation of Matt Drudge, and his news portal, “The Drudge Report,” which would soon have millions of visitors each day, became a model of how much influence a single individual could have using the Web.
For the next two chapters we will look at the practical craft of constructing compelling phrases, sentences and narratives. Think of it as everything you were supposed to learn in language and grammar classes in primary and secondary school— but in only a few thousand words. This time around, we're going to teach you the actual stuff you need to be a professional writer, and make a living doing so.
For that reason, we aren't going to spend time on grammar: in the real world, the only rule of grammar is that it works in getting the message through to the reader. Nor are we going to discuss rhetoric, other than, once again, what works. Nor vocabulary: experience has shown that working every day as a writer will force you to expand your vocabulary if you are going to be able to effectively explain yourself. Nor punctuation: not least because that field has become increasingly fluid in recent years.
Rather, we are going to use these two chapters to look at how you use language in the most powerful way to capture and hold readers, enhance their emotional response to what you've written, and keep them reading through to the end. Compared to that, whether you've written a sentence fragment or split an infinitive is inconsequential. Indeed, in professional practice, there are good arguments to be made for both. So, let's begin our short course in Real- Life Writing— or, as the author prefers, Writing for Money.
The parts of language and their roles
Sounds are senses
Once, during a safari in Namibia, the author met a man, a tracker and guide, who was one of the most linguistically accomplished individuals on Earth. His father was a member of the Ovambo tribe, the dominant African people in that part of sub- Saharan Africa; his mother was Bushman, a member of the San people. Meanwhile, by the nature of his work, and his own native language skills, this man dealt regularly with people from around the world and had to learn to speak with them in their own languages.
You can't write anything if you don't have anything to write about.
That observation may sound stupid and obvious, but you'd be amazed how many poets, columnists, feature writers and novelists try to force words onto the page without any real knowledge of what they are writing about. Some are even reduced to making up “facts” and sources that aren't real. Such are the demands of money, deadline and ambition. Sometimes they pull it off; but not for long.
Real, honest writing— even if it is fiction— requires real, honest information on what you are writing about. And the only way to get that information is to go out looking for it. That means interviewing sources or eyewitnesses, or visiting the sites of key events, or digging deep into official records, or searching far out into the hinterlands of the Internet. The closer you can get to actual participants or witnesses to the event you wish to describe, including the documents they leave behind— the more legally certified the documents you find, and the more verified and cross- referenced the file you find— the better off you are going to be.
Why do this? Because you owe it to yourself as a professional to get things right. And because you owe it to your readers, your client or your employer not to mislead them or place them in legal or financial jeopardy. Sound overdramatic? Wait until you get a story wrong.
We once worked for an editor who was the very model of a conscientious reporter— including checking and double- checking every factual claim made by his staff of young reporters. At first we thought he was overly careful, at the expense of stripping some of the power out of our stories. Then he told us of an experience from his own days as a young reporter.
It seems that, while still little more than a rookie, he wrote a profile of a fast- growing new company that had a hot new product, skyrocketing sales and the prospect of even better days ahead.
All writing is colored by the medium in which it appears, but with television and radio news, the writing is largely defined by the nature of those media. With television, almost all writing is subordinate to the visual imagery available, and the words almost always precede the image. As a result, television news reporting (as compared to print reporting) is typically more of a headline- and- caption production. In radio, the limitations are almost always due to time, as the spoken word is much slower than the read word. Also, in radio any nonverbal sounds (from real- life recordings to sound effects) are almost always subordinate to the spoken word— or used as a set- up to those words.
Why pursue a TV or radio news- writing career?
One reason is that television and radio news— especially the former— are just more glamorous than print. That's because an on- air face or voice is much more individualized than a simple print byline. Still, as the years pass, the three media are growing closer together as the Web enables television and radio reporters to write in a longer format; and print reporters to go on- air with Web videos.
Television and radio news also typically reaches a larger audience than print, especially in an era when newspapers are fading and dropping editions, and television and radio news is extending across the 24- hour cycle. Also, because of the delays created by press runs, TV and (even more) radio news is typically more timely, with updates and breaking news presented throughout the day.
But perhaps the most important reason for pursuing television or radio journalism over print is the personal connection you make with your audience. This is especially true with television, where viewers can come to believe they actually know you— and will sometimes come up and talk to you in public places as if you are an old friend. That can be irritating, but if you have the right, outgoing, personality becoming a local celebrity can be an added reward. That kind of attention almost never happens with print reporters, even popular columnists.
Every human being knows the definition of rejection in his or her life. “Rejection” in the context of professional writing is the experience of having a proposal or a completed piece of writing turned down by a targeted publication or other media venue.
Like other forms of rejection, literary rejection can be a deeply painful experience, particularly when you take that rejection as a judgment on your writing skills or, worse, your value as a human being. The key is to put that rejection in proper perspective and develop strategies for turning it into acceptance. In this chapter we'll look at those strategies in detail.
(For simplicity, we use “publication” to represent all media.)
Basic rules
We start with some simple (but not necessarily easy) rules for responding to a rejection letter or e- mail from a publication.
Learn to deal with rejection. It's part of being a writer, as much as it is of acting, dancing and other arts. Want to know real rejection? Have a career as an actor: endless casting calls and tryouts, almost always ending in rejection, sometimes on the most personal terms. Compared to that, having your writing rejected is easy. Just tell yourself up- front that rejection goes with being a writer— even famous writers sometimes fail to sell a piece— and that you need to incorporate that experience into your working life.
Get over it. Having an article or story or book rejected is usually not an evaluation of you as a person, or even of your talent, but simply a bad fit between what you wrote and what the publication needs. Given that, why take it personally? Instead, find the publication or publisher where it is a good fit.
Get back to work. Get angry. Get over it. And get on to the next project or pitch. Don't dwell on rejection— it'll break your spirit and make you hesitate to pitch the next time. Great writing is built on confidence.
Reporting the news is the heart of journalism, which is why it has always constituted the front page of newspapers, is the primary work of wire services and is the cornerstone of journalism education. News reporters make up the plurality of nearly all newsrooms. All other types of reporting are second in priority to the news.
What is news?
It is in the word itself. “News” is made up of events of importance to the reader that have either just occurred or are still unfolding. Reporting on these events typically requires the reporter to be on the scene (if possible), gathering facts and quotes and preparing a story under a tight deadline. In television and radio this reporting often is done live, with the reporter standing before a camera or microphone and improvising the story from rough notes.
Types of news reporting
In terms of format, all news reporting is basically the same— as we shall see. Where news reporting is differentiated is by beat; that is, by the particular area of daily life where news stories are most likely to be generated. Here is a list of some of the most common beats for news reporters:
General Assignment— This is often the first job of the neophyte reporter. As the name suggests, general assignment reporters arrive on the job each day and are told by the assignment editor what story they are to cover. The stories can come from any one of the various beats, even from one of the other departments of the publication or station. For newcomers, general- assignment reporting is sort of a test by fire: everyday is something new and often unpredictable. One day you can be interviewing a famous figure visiting your town, the next covering the horror of a plane crash.
For veteran reporters, general assignment offers both novelty and a kind of completeness. The author used to work in a newspaper newsroom across from two women, both veteran reporters who had chosen to stay in general assignment work. Because I was a business reporter, my desk was perpetually buried in press releases and briefing documents related to ongoing stories. By comparison, when the two ladies finished their work day and filed their stories, they would just sweep all the paperwork off their desks into a nearby waste basket and go home without a worry.
Until now we have been largely discussing writing careers in the commercial world. In this chapter, we'll take a break from that thread and look at an alternative writing career— one that may not carry the same potential for fame, but can actually be more financially rewarding, stable and fulfilling than what we think of as “professional writing”: academic writing.
Like many writers, the author (without really knowing it) reached a decision point in his writing career in the months before his undergraduate college graduation. In one direction stretched the commercial world, in the other, graduate school and a life in academia.
I barely gave it a thought: I wanted to be a writer, not a professor. And, so, I went to work in corporate public relations and after four years became a newspaperman. I never looked back until decades later, when I finally did become an adjunct professor. And it was only then, as I looked at the life of a college professor up close— and even more when I finally identified the sophisticated parallel universe of academic publishing— that I began to wonder if I had made a terrible mistake all those years before.
The author will try to present this other world as objectively as possible, though you may notice a bit of bias— a combination of envy and amusement— creeping in.
Definition of the academic track
The academic track is the pathway by which writers establish a career by positioning themselves within academia and build a reputation— and earn awards— by writing for audiences and publishers largely within the academic world. The biggest difference between commercial writing and academic writing is that the former is typically the career itself, while the latter is a valuable sidelight to the primary work of teaching.
The academic pathway
If a commercial writing career can best be described as the zig- zagging pursuit of ever- greater opportunity, an academic writing career is much more of a straight, predetermined path where the only thing in doubt is when (and if) the next promotion will occur. The standard path, pretty much set in stone two centuries ago, goes like this.
• identify the key components of a DBMS architecture;
• understand how these components work together for data storage, processing, and management;
• categorize DBMSs based upon data model, degree of simultaneous access, architecture, and usage.
Opening Scenario
To kick-start its business, Sober purchased the customer database of Mellow Cab, a firm that recently stepped out of the taxi business. Unfortunately, the database has been handed over in a legacy CODASYL format that Sober is not familiar with. Sober also needs a new database to store transaction details whenever passengers book either a ride-hailing or ride-sharing service. Other data (e.g., multimedia) are an option they are interested in. Sober wants to continuously store the location of its taxis and periodically review hot-spot pick-up and drop-off locations. Sober is looking at ways to manage all these data sources in the optimal way.
As discussed in Chapter 1, a DBMS supports the creation, usage, and maintenance of a database. It consists of several modules, each with their specific functionality, that work together according to a predefined architecture. In this chapter, we zoom into this internal architecture and provide a categorization of DBMSs along various dimensions. The overview of the chapter is straightforward. We start by discussing the components that together make up a DBMS. Next, we provide a classification of DBMSs in terms of data model, degree of simultaneous access, architecture, and usage.
Architecture of a DBMS
As discussed before, a DBMS needs to support various types of data management-related activities, such as querying and storage. It also must provide interfaces to its environment. To achieve both of these goals, a DBMS is composed of various interacting modules that together make up the database management system architecture. Figure 2.1 shows an overview of the key components of a DBMS architecture. We review each component in more detail in what follows.
Figure 2.1 is by no means exhaustive. Depending upon the vendor and implementation, some components may be left out and others added. On the left, you can see various ways of interacting with the DBMS. DDL statements create data definitions that are stored in the catalog. Interactivequeries are typically executed from a front-end tool, such as a command-line interface, simple graphical user interface, or forms-based interface. Applications interact with the DBMS using embedded DML statements.
• understand the role and importance of catalogs, metadata, data quality, and data governance;
• identify key roles in database modeling and management;
• understand the differences between an information architect, database designer, data owner, data steward, database administrator, and data scientist.
Opening Scenario
Sober realizes that the success of its entire business model depends on data, and it wants to make sure the data are managed in an optimal way. The company is looking at how to organize proper data management and wondering about the corresponding job profiles to hire. The challenge is twofold. On the one hand, Sober wants to have the right data management team to ensure optimal data quality. On the other hand, Sober only has a limited budget to build that team.
In this chapter, we zoom into the organizational aspects of data management. We begin by elaborating on data management and review the essential role of catalogs and metadata, both of which were introduced in Chapter 1. We then discuss metadata modeling, which essentially follows a similar database design process to that which we described in Chapter 3. Data quality is also extensively covered in terms of both its importance and its underlying dimensions. Next, we introduce data governance as a corporate culture to safeguard data quality. We conclude by reviewing various roles in data modeling and management, such as information architect, database designer, data owner, data steward, database administrator, and data scientist.
Data Management
Data management entails the proper management of data and the corresponding data definitions or metadata. It aims at ensuring that (meta-)data are of good quality and thus a key resource for effective and efficient managerial decision-making. In the following subsections we first review catalogs, the role of metadata, and the modeling thereof. This is followed by a discussion on data quality and data governance.
Catalogs and the Role of Metadata
The importance of good metadata management cannot be understated. In the past this was often neglected, resulting in significant problems when applications needed to be updated or maintained. In the file-based approach to data management, the metadata were stored in each application separately, creating the issues discussed in Chapter 1.
• understand the differences between the file versus database approach to data management;
• discern the key elements of a database system;
• identify the advantages of database systems and database management.
Opening Scenario
Since Sober is a startup company, it must carefully decide how it will manage all its data. The company is thinking about storing all its data in Word documents, Excel files, and maybe some other files (e.g., Notepad) as well.
In this chapter, we discuss the fundamental concepts of database management. Many ideas presented here are elaborated in later chapters. We kick off by reviewing popular applications of database technology, and follow this by defining key concepts such as a database and a database management system, or DBMS. Next, we step back in time and discuss the file-based approach and contrast it with the database approach to data management. We then zoom into the elements of a database system. We conclude by discussing the advantages of database design.
Applications of Database Technology
Data are everywhere and come in different shapes and volumes. These data need to be stored and managed using appropriate data management or database technologies. Think about the storage and retrieval of traditional numeric and alphanumeric data in an application developed to keep track of the number of products in stock. For each product, the product number, product name, and available quantity needs to be stored. Replenishment orders need to be issued as soon as the quantity drops below the safety limit. Every replenishment order has an order number, order date, supplier number, supplier name, and a set of product numbers, names, and quantities.
Database technology is not just for traditional numeric and alphanumeric data. It can also store multimedia data such as pictures, audio, or video – YouTube and Spotify support the querying of music based upon artist, album, genre, playlist, or record label. Biometric data, including fingerprints and retina scans, are often used for security, such as border control as you enter a country. Information is also gathered by wearables, such as a Fitbit or an Apple Watch, which continuously monitor and analyze your health and fitness. Geographical information systems (GIS) applications, such as Google Maps, store and retrieve all types of spatial or geographical data.
• what is meant by “NoSQL” and how this movement differs from earlier approaches;
• to understand the differences between key–value-, tuple-, document-, column-, and graph-based databases;
• about the defining characteristics of NoSQL databases, such as their capability to scale horizontally, their approach toward data replication and how this relates to eventual consistency, their APIs, and how they are queried and interacted with;
• how to query NoSQL databases, using MapReduce, Cypher, and other approaches.
Sober has been happily using its relational database system for quite a while now, but is experiencing some limits of its existing system. In particular, due to the growth of its mobile applications (now serving many users simultaneously), some queries are performing slower, causing users to wait a few seconds when working with the app. Sober's database administrators have identified that this is mainly due to the normalized approach of its relational database model, causing many queries to involve joining tables, which results in bottlenecks when thousands of users are hailing cabs simultaneously.
Another issue is that the mobile developers regularly want to experiment with new features (to do so, a “beta” version of the app is rolled out to a subset of users), though given the emphasis on data consistency put forward by RDBMSs, changing existing data schemas so they can handle new data fields is a time-intensive effort. The app developers are not very familiar with this, causing back-and-forth between the development team and the database administrators.
Given these new issues, Sober is thinking of different scenarios to handle its growth. One option would be to roll out additional servers to handle the increased usage, but keep using the RDBMS system to do so. Another possibility would be to explore a NoSQL database solution. Sober's database administrator has heard about newer approaches such as MongoDB, Cassandra, and others, which offer strong scaling opportunities and a schema-free approach to data management.
Relational database management systems (RDBMSs) put a lot of emphasis on keeping data consistent. They require a formal database schema, and new data or modifications to existing data are not accepted unless they comply with this schema in terms of data type, referential integrity, etc. Sometimes this focus on consistency may become a burden, because it induces (in some cases unnecessarily) overhead and hampers scalability and flexibility.
• grasp the basics of distributed systems and distributed databases;
• discern key architectural implications of distributed databases;
• understand the impact of fragmentation, allocation, and replication;
• identify different types of transparency;
• understand the steps in distributed query processing;
• understand distributed transaction management and concurrency control;
• grasp the impact of eventual consistency and BASE transactions.
Opening Scenario
As Sober envisions growing as part of its long-term strategy, it wants to have a careful understanding of the data implications involved. More specifically, the company wants to know if it would make sense to distribute its data across a network of offices and work with a distributed database. Sober wants to know the impact of data distribution on query processing and optimization, transaction management, and concurrency control.
In this chapter, we focus on the specifics of distributed databases (i.e., systems in which the data and DBMS functionality are distributed over different nodes or locations on a network). First, we discuss the general properties of distributed systems and offer an overview of some architectural variants of distributed database systems. Then, we tackle the different ways of distributing data over nodes in a network, including the possibility of data replication. We also focus on the degree to which the data distribution can be made transparent to applications and users. Then, we discuss the complexity of query processing and query optimization in a distributed setting. A next section is dedicated to distributed transaction management and concurrency control, focusing on both tightly coupled and loosely coupled settings. The last section overviews the particularities of transaction management in Big Data and NoSQL databases, which are often distributed in a cluster set-up, presenting BASE transactions as an alternative to the traditional ACID transaction paradigms.
Distributed Systems and Distributed Databases
Ever since the early days of computing, which were dominated by monolithic mainframes, distributed systems have had their place in the ICT landscape. A distributed computing system consists of several processing units or nodes with a certain level of autonomy, which are interconnected by a network and which cooperatively perform complex tasks. These complex tasks can be divided into subtasks as performed by the individual nodes.