In computer science and engineering, data structure and algorithm are two very important parts. Data structure is the logical representation of data, so that insertion, deletion, and retrieval can be done efficiently, and an algorithm is the step-by-step procedure to solve any problem. By studying different data structures, we are able to know their merits and demerits, which enriches our knowledge and our ability to apply the appropriate data structures at proper places when we try to write new applications. Studying different standard algorithms provides us much knowledge about solving new problems. Both data structures and algorithms are interrelated and are complementary to each other. By studying both data structures and algorithms, we may acquire a solid foundation of writing good code. This comprehensive knowledge helps to understand new frameworks as well.

With the studying of data structures and algorithms, it is very important to implement them using proper languages. Several books have been written on this topic using the C language. But today Python has become very popular because of its features such as being easy, open source, object oriented, portable, multi-threaded, extensive libraries, embeddable, etc. Hence, in this book data structures and algorithms are implemented using Python.

About the Book

This book is written to serve the purposes of a textbook for undergraduate courses of computer science and information technology and for undergraduate and postgraduate courses of computer application where data structure is one of the subjects in the syllabus. In this book different data structures and algorithms are discussed in a lucid manner so that students can understand the concept easily. All the relevant data structures and their operations are discussed with diagrams and examples for better understanding. After discussing relevant algorithms in detail, the algorithmic representation and the corresponding code in Python are given. Various programming examples along with new problems for practice and a set of MCQ questions at the end of each chapter increase the self-learning process of the students.

A stack is a very important data structure. We find its application in various situations in computer science. In this chapter we will discuss the operations related to stacks, how a stack is represented in memory, various areas of applications where stacks are used, etc.

8.1 Definitions and Concept

A stack is a linear data structure in which both insertion and deletion operations occur only at one end. The insertion operation is commonly known as PUSH and the deletion operation is known as POP. As both the operations occur at one end, when the elements are pushed into a stack the elements are stored one after another and when the elements are popped only the topmost element can be removed first and then the next element gets the scope to be popped. In real life also we can see various analogies of this structure, such as a stack of coins, stack of plates, stack of boxes, etc. (Figure 8.1). Suppose, in a stack of coins, if we want to place a new coin, it should be placed at the top of the stack, and if we want to get a coin from that stack, we should remove the topmost coin from the stack. Since in a stack always the element that was inserted last is removed first, it is known as a LIFO (Last In First Out) data structure.

8.2 Operations Associated with Stacks

The basic operations related to a stack are the following:.

• • Push: Insert an element into a stack.

• • Pop: Retrieve an element from a stack.

Apart from these two basic operations, to use a stack efficiently we need to add the following functionalities:

• • Peek: Get the top element of the stack without removing it.

• • Isempty: To check whether the stack is empty or not.

• • Isfull: To check whether the stack is full or not.

All these operations execute with the time complexity O(1).

In some situations we need to check the top element without removing it. That is why the peek operation is required. Again, if a stack is full, the push operation cannot be done on the stack. Similarly, pop and peek operations cannot be performed on an empty stack. Thus, before these operations we need to check whether the stack is full or empty.

Our basic aim is to write a good program. A program must give correct results. But a correct program may not be a good program. It should possess some characteristics such as readability, well documented, easy to debug and modify, etc. But the most important feature is that it should efficiently. Efficient means that the program should take minimum time and minimum space to execute. To achieve this, we need to store and retrieve data in memory following some logical models. These are called data structures.

1.1 Concept of Data Type

Before discussing data structures let us recapitulate what data are and what data type is. Data are raw facts. Through a program we convert data into our required information. For example, marks scored by a student in some examination are data. But total marks, average marks, grade, whether the student has passed or failed – these are information. The nature of all data is not the same. Some of them are whole numbers, some are real numbers that contain some fractional value, some are of character type, etc. These types of data are known as data type. Data type indicates the nature of the data stored in a particular variable. Different programming languages support different data types. Some are common, some are different. Integer, character, float, double, etc., are examples of data types. We can group data types into two categories. One is primitive or basic data type and the other is user defined data type.

1.1.1 Primitive data type

The data types which are predefined and in-built are known as primitive or basic data type. In Python, the primitive data types are number, string, and Boolean. Number data types are of three types in Python. These are integers, floating-point numbers and complex numbers. Strings are collections of characters. There are several types of strings in Python. These are escape sequences, raw strings, and triple quoted strings. Boolean data type represents only two values: True and False. There is another basic data type, named None, which represents the absence of any value.

Chapter 1: Data Structure Preliminaries

Chapter 2: Introduction to Algorithm

Chapter 3: Array

Chapter 4: Python Data Structures

Chapter 5: Strings

Chapter 6: Recursion

Chapter 7: Linked List

Chapter 8: Stack

Chapter 9: Queue

Chapter 10: Trees

Chapter 11: Heap

Chapter 12: Graph

Chapter 13: Searching and Sorting

Chapter 14: Hashing

In the previous chapter we have studied about arrays. An array is a widely used data structure in almost every high level programming languages. In this chapter we will discuss some very useful data structures of Python. These are lists, tuples, sets, and dictionaries. All these data structures are of type compound data structures since they consist of some basic or primitive data structure/data type.

4.1 Lists

List is a very important and useful data structure in Python. It has almost all the functionalities of an array but with more flexibility. Basically a list is a collection of heterogeneous elements. List items are ordered, which means elements of a list are stored in a specific index position. List items are mutable, which indicates that it is possible to change or edit the elements of a list, and duplicate items are allowed in a list. Another important feature of a list is that it is dynamic. It can grow or shrink during program execution according to our requirement.

4.1.1 Creating a List

A list is a collection of elements that are separated by commas (,) inside a pair of square brackets. Hence we can create a list by putting comma separated elements within square brackets. The general format to defining a list is:

Hence we may create a list of integers as:

A list of string can be created as:

A list with heterogeneous data can be created as:

An empty list can be created as:

An empty list can also be created using a constructor as:

4.1.2 Accessing List Elements

List elements are accessed just like array elements, i.e., the elements of a list are accessed using a list index. Like arrays, in case of lists also the index value starts from 0. Hence, the first position of a list is 0, second is 1, and so on. To access a list element we need to mention the list name followed by the list index enclosed within [ ].

Another important data structure is queue. Queues are also exhaustively used in various situations in computer science. In this chapter we will discuss the operations related to queues; how a queue is represented in memory, both array and linked representation of queues, different types of queues and various applications of queues.

9.1 Definitions and Concept

A queue is a linear data structure in which insertion and deletion operations take place at two different ends. The insertion operation is commonly known as ENQUEUE and the deletion operation is known as DEQUEUE. These two operations take place at two different ends. The end at which elements are inserted is known as the rear end and the other end at which elements are deleted from the queue is known as the front end. Elements are inserted through the rear end and removed through the front end following the First-In-First-Out (FIFO) order, i.e. in the order the elements entered the queue they will leave the queue maintaining the same order. In real life we are very familiar with queues. We can see several queues in our surroundings. For example, passengers standing in front of a railway ticket booking counter. The passenger who stands at the first position gets the ticket first and when a new passenger comes, he joins the queue at the end. The same can be seen when passengers wait at a bus stop, people standing in front of a ticket counter in a cinema hall, luggage moving through conveyor belt, cars passing through single lane in a one-way road, etc.

9.2 Operations Associated with Queues

The basic operations related to a queue are as follows:

• • Enqueue: Insert an element into a queue.

• • Dequeue: Retrieve an element from a queue.

Apart from these basic two operations, to use a queue efficiently we need to add the following functionalities:

• • Peek: Get the front element of a queue without removing it.

• • Isempty: To check whether a queue is empty or not.

• • Isfull: To check whether a queue is full or not.

All these operations execute with the time complexity O(1).

Like in stack, the peek operation is required to check the front element of the queue without removing it. And if a queue is full, the enqueue operation cannot be done on the queue. Similarly, dequeue and peek operations cannot be performed on an empty queue.

Recursion is a very important concept in programming technique. In programming there are situations where we may find that the solution of a problem can be achieved by solving smaller instances of that same problem. For example, in tree data structure, all the properties that comprise a tree also feature in its sub-trees. So, to get a solution for the tree first we need to get the solution of its sub-trees. Again to get the solution of a sub-tree we need to get the solution of the sub-trees of this sub-tree, and so on. This is the basis of recursion. This concept helps to solve many complex problems very easily.

6.1 Definition

In almost every programming language, a function or procedure may invoke other functions or procedures. Not only from other functions, a function may be invoked from within itself as well. When a function is invoked by itself, the function is called a recursive function. And the process is called recursion. Instead of direct invocation, it may happen that the first function invokes the second function and the second function again invokes the first function in turn. This is also recursion. So, we can define recursion as the process where a function is directly or indirectly invoked by itself. Though it is a comparatively more memory consuming process, it is very effective in implementation where the algorithm demands the same repetitive process but use of iteration is difficult.

For example, we can define factorial of n as n! = n * (n-1)!. So, to find the factorial of n we need to calculate factorial of n-1 first. Again, to calculate the factorial of n-1 we need to calculate factorial of n-2, and so on. But this process may not continue indefinitely. Stack will be overflowed in that case. Hence, we must impose some criteria based on which this invocation should be restricted. This is called base criteria or base condition. Thus, every recursive function must have a base condition. In the above example, we may set the base condition as: when the value of n is 0, instead of recursive call it directly returns 1.

In the previous two chapters we have studied about arrays and other in-built data structures like lists, tuples, sets and dictionaries. In this chapter we will discuss another in-built data type/data structure of Python, and that is string. In programming languages, string handling is a very important task and in data structure we will learn about different algorithms for efficient string handling.

5.1 Introduction

A string is a sequence of characters allocated contiguously in memory where the characters may be letters of the alphabet, digits, punctuation marks, white spaces or any other symbols. In Python a string is implemented as an object of the in-built class str. To define a string constant we may use single quotation (‘ ’), double quotation (“ ”) or triple quotation (‘“ ”’). So, ‘India’, “India” and ‘“India”’ – all three are valid string constants in Python. Apart from a normal string, there are two other types of strings. These are escape sequences and raw strings. Like C, C++ and Java programming, Python also supports escape sequences. An escape sequence is the combination of a backslash (\) and another character that together represent a specific meaning escaping from their original meaning when used individually. Table 5.1 shows some escape sequences in Python.

The following example shows the use of escape sequences:

Example 5.1: Example to show the use of escape sequences.

print(“Hello \nClass!!”)

Output:

Hello

Class!!

print(“Hello \tClass!!”)

Output:

Hello Class!!

print(‘Cat\rM’)

Output:

Mat

print(‘Cat\b\bu’)

Output:

Cut

print(“Welcome to the world of \‘Python\’”)

Output:

Welcome to the world of ‘Python’

print(“\”India\” is great.”)

Output:

“India” is great.

If a string is preceded by an ‘r’ or ‘R’, then it is called a raw string. Within a raw string, escape sequence do not work. They behave like a normal character. Consider the following example:

Example 5.2: Example to show the use of raw string.

print(‘Techno\nIndia’)

Output:

Techno

India

print(r‘Techno\nIndia’)

Output:

Techno\nIndia

print(‘Techno\\India’)

Output:

Techno\India

print(R‘Techno\\India’)

Output:

Techno\\India

5.2 Basic String Operations

Strings are immutable in Python. Hence, we cannot modify a string. But we can create a new string after manipulating an old string. We can extract an individual character or portion of a string. Characters within a string are accessed using the subscript operator and the index value.

In this chapter we will discuss an important data structure: a heap. Though there are several variations of this data structure, such as binary heap, d-ary heap, B heap, Fibonacci heap, etc., we will discuss here the most popular heap data structure – the binary heap. In the following sections we will mention heap to indicate the binary heap.

11.1 Definition and Concept

Heap is a very important tree-based data structure. It is widely used in computer science. We can define a heap as a binary tree that has two properties. These are: shape property and order property. By shape property, a heap must be a complete binary tree. By order property, there are two types of heaps. One is max heap and the other is min heap. By default, a heap means it is a max heap. In max heap, the root should be larger than or equal to its children. There is no order in between the children. This is true also for its sub-trees. In min heap the order is the reverse. Here the root is smaller than or equal to any of its children. Thus the root of a max heap always provides the largest element of a list whereas the root of a min heap always provides the smallest element. As a heap is a complete binary tree, its maximum height is O(log2n) where n is the total number of elements. Thus both the insertion of new node and the deletion of an existing node can be done in O(log2n) time. Figure 11.1 shows a max heap and a min heap.

11.2 Representation of a Heap in Memory

Like other data structures, a heap also can be represented in memory using an array (or a list in Python) and a linked list. The main problem in tree representation using arrays is in wastage of memory. But in case of complete binary tree representation, that chance is almost nil. Moreover, we need not waste space for storing references of left and right children. So, using arrays (or lists in Python) we can represent the heap data structure efficiently.

In the previous chapter we have discussed three different searching techniques. Linear search works on both ordered and unordered data but time complexity is O(n). Binary search works on only ordered data but time complexity is O(log2n) which is much faster than linear search. Interpolation search generally works faster than binary search and its time complexity is O(log2(log2n)) when elements are evenly distributed. But there is another application where, irrespective of the size of the array/list, a search operation can be performed with time complexity O(1), i.e. in constant time. This is possible due to hashing. In this chapter we will discuss this.

14.1 Definitions and Concept

Hashing is a procedure by which we can store or retrieve data in constant time, i.e. with time complexity O(1). To achieve O(1) time complexity we may follow a simple process. Suppose we have to store the data of students whose roll numbers range from 1 to 60. For this purpose we may take an array/list and store the data of each student at the corresponding index position which matches with the roll number. In this situation, to access a student, if we know his/her roll number, we can directly access his/her data. For example, if we want to read the data of the student whose roll number is 27, we may directly access the 27th index position of the array/list. But the problem is that in real life key values are not always as simple. You may find that in a certain university someone's roll number may be 21152122027. This does not imply that it is a sequential number, nor that this number of students are admitted in a year in that university. It may be a nomenclature of a key value where the first two digits may denote the year of registration, the next three digits are the college code, the next two digits may indicate stream, and so on. This is true for not only roll numbers but also any other key value. Thus it is clear that the number of digits does not indicate the total number of elements. In the above example hardly 10,000 or 20,000 students may take admission in that university in a year.

Algorithm is a very common word in computer science, especially in case of any procedural programming languages. In this chapter we will know about algorithms, different types of algorithms, different approaches to designing an algorithm, analysis of algorithms, etc. We will also be able to learn how an algorithm can be written using different control structures.

2.1 What Is an Algorithm?

We can define an algorithm as a step-by-step procedure to solve a problem. This is much like a recipe for cooking an item. To cook a new item, we follow the instructions one by one as given in its recipe. Similarly, to write a program we need to follow the algorithm. Once we are able to generate the algorithm of a problem, then writing its code is not a huge task at all. But before writing an algorithm we have to keep in mind that any algorithm must have the following characteristics:

• • Input: Inputs are the values that are supplied externally. Inputs are those without which we cannot proceed with the problems. Every algorithm must have zero or any number of inputs.

• • Output: Output is the outcome of an algorithm. Every algorithm produces at least one output.

• • Definiteness: All the instructions in the algorithm must be clear and should not be ambiguous.

• • Finiteness: Whatever may be the inputs for all possible values, every algorithm must terminate after executing a finite number of steps.

• • Effectiveness: All the instructions in the algorithm should be very basic so that every instruction of the algorithm can be converted to programming instruction easily. Effectiveness indicates that every instruction is a step towards the solution.

• • Feasible: Every instruction must be a feasible instruction.

2.2 Importance of an Algorithm

When we try to write a program, first we need to identify all the tasks. Some tasks are easily identifiable while some tasks maybe hidden within the problem definition. For that a detailed analysis of the problem is required. After proper analysis we are able to find what the expected output is. What are the inputs required using which we can get the output? Using the inputs how we reach the solution? To accomplish this ‘how’, after identifying all the tasks we need to arrange them in a proper sequence.

So far we have discussed different linear data structures like arrays, linked lists, stacks, queues, etc. These are called linear data structures because elements are arranged in linear fashion, i.e. one after another. Apart from these there are also some non-linear data structures. Trees and graphs are the most common non-linear data structures. In this chapter we will discuss the tree data structure. A tree structure is mainly used to store data items that have a hierarchical relationship among them. In this chapter we will discuss the different types of trees, their representation in memory, and recursive and non-recursive implementation.

10.1 Definition and Concept

A tree is a non-linear data structure that is used to represent hierarchical relationship among data items. Basically a tree is an acyclic and connected graph. It consists of some nodes and these nodes are connected by edges. The topmost node or starting node is known as root. Zero or more nodes can be connected with this root node through edges. The structure of a tree is recursive by its nature. Each node connected with the root node may be further considered as a root node with which some other nodes can be connected to form a sub-tree. Thus, a tree, T, can be defined as a finite non-empty set of elements among whose one is root and others are partitioned into trees known as sub-trees of T. Figure 10.1 shows a sample tree structure.

10.2 Terminology

In this section we will know about the basic terminologies of trees considering Figure 10.1.

Node: It is the main component of a tree. It contains the data elements and the links to other nodes. In the above tree, A, B, C, D, etc. are the nodes.

Root Node: The topmost node or starting node is known as the root node. Here A is the root node.

Edge: It is the connecter of two nodes. In Figure 10.1, the line between any two nodes can be considered as an edge. As every two nodes are connected by a single edge only, there should be exactly n-1 edges in a tree having n nodes.

We have already learned that to store a set of elements we have two options: array and Python's in-built list. Both provide easy and direct access to the individual elements. But when we handle a large set of elements there are some disadvantages with them. First of all, the allocation of an array is static. Thus we need to mention the size of the array before compilation and this size cannot be changed throughout the program. On the other hand, Python list may expand according to requirement but that does not come without a cost. Adding new elements into a list may require allocations of new block of memory into which the elements of the original list have to be copied. Again, if a very large array is required to be defined, then the program may face a problem of allocation. As an array is a contiguous allocation, for very large sized arrays, there may be a situation where even though the total free space is much larger than required, an array cannot be defined only due to those free spaces not being contiguous. Another problem related to arrays and lists is in insertion and deletion operations. If we want to insert an item at the front or at any intermediate position, it cannot be done very easily as the existing elements need to be shifted to make room. The same problem is faced in case of deleting an item. The solution to all these problems is to create a Linked List.

In a linked list we need not allocate memory for the entire list. Here only a single node is allocated at a time. This allocation takes place at runtime and as and when required. Thus, unlike an array, a large contiguous memory chunk is not allocated; instead very small memory pieces are allocated. When we need to store the first item, we have to allocate space for it only and it will store the data at this place. Next when the second item needs to be stored, we will again allocate space for the second item, and so on. As these memory spaces are not contiguous, to keep track of the memory addresses, along with the data part we have to store an extra piece information with each item and it is the address of the next item.

In this chapter we will discuss two important operations in programming languages. These are searching and sorting. There are several algorithms for both searching and sorting. Every algorithm has some merits and demerits. In this chapter we will discuss some important searching algorithms as well as sorting algorithms. We will discuss the working principles of these algorithms in detail and how they can be implemented in Python, we will derive their time complexity, and finally we will compare them with each other.

13.1 Introduction to Searching

Searching is a process by which we can check whether an element is present within a set of elements or not. If it is present, we can say that the search operation is ‘Successful’ and this process also find the position of the element in the list. But if the element is not present, the search operation is considered as ‘Failure’ and the process terminates with an appropriate failure message. In traditional programming languages searching operations are generally implemented using arrays. In Python we can implement searching algorithms using arrays as well as in-built list data structures. There are several searching algorithms to search for an element from an array/list. Here we will discuss the three most common searching algorithms.

• • Linear Search

• • Binary Search

• • Interpolation Search

Among the above three searching algorithms, to implement linear search the array/list need not be sorted necessarily. But the initial requirement for the other two algorithms is that the array/list must be sorted.

13.1.1 Linear Search

The algorithm which checks each element of a list starting from the first position is known as Linear Search. As this algorithm checks each element in a list sequentially it is also known as Sequential Search. When the elements in a list are unsorted, i.e. not in proper order, this searching technique is used. Suppose we have the following list of elements:

And the key element is 40, i.e. we want to search whether 40 is present in the list or not and if it is present, then we also need to know its position in the list. In case of linear search, we start checking from the beginning of the list, i.e. from the index position 0.