In various applications of computer vision and imageprocessing, it is required to detect points in animage, which characterize the visual content of thescene in its neighborhood and are distinguishableeven in other imaging instances of the same scene.These points are called key points of an image andthey are characterized by the functionaldistributions, such as distribution of brightnessvalues or color values, around its neighborhood foran image. For example, in the monocular and stereocamera geometries, various analyses involvecomputations of transformation matrices such as,homography between two scenes, fundamental matrixbetween two images of the same scene in a stereoimaging setup, etc. These transformation matricesare computed using key points of the same scenepoint of a pair of images. The image points of thesame scene point in different images of the sceneare called points ofcorrespondence or corresponding points. Key points ofimages are good candidates to form such pairs ofcorresponding points between two images of the samescene. Hence detection and matching of key points ina pair of images are fundamental tasks for suchgeometric analysis.

Consider Fig. 4.1, where images of the same scene arecaptured from two different views. Though theregions of structures in the images visuallycorrespond to each other, it is difficult toprecisely define points of correspondences betweenthem. Even an image of a two-dimensional (2-D)scene, such as 2-D objects on a plane, may gothrough various kinds of transformations, likerotation, scale, shear, etc. It may be required tocompute this transformation among such a pair ofimages. This is also a common problem of imageregistration.

Color is a psycho-physiological property of humanvisual experiences when the eyes look at objects andlight. Color is not a physical property of thoseobjects or light, rather, it is the result of aninteraction between physical light in theenvironment and human visual system (Palmer, 1999).For processing color images, it is required todevelop an understanding on how colors arerepresented following human perception.

3.1 Light sources

A broad range of electromagnetic spectrum, shown inFig. 3.1, consists of electromagnetic waves rangingfrom very long wavelengths at radio waves to veryhigh frequency at gamma waves. A very narrowinterval in this spectrum, toward the higher end ofspectral frequencies, accounts for the visible raysand it is called the visiblespectrum. The light and colors that ahuman eye perceives relate to the frequencies ofwaves that fall under the visible spectrum. Apictorial representation of the correspondence ofwavelengths in the visible range of the spectrum todifferent perceived colors has been shown in Fig.3.1. There are seven distinguishable colors in thefigure, violet, indigo, blue, green, yellow, orange,and red, usually known in order of their increasingwavelengths by the acronym of VIBGYOR. The luminancesensitivity function that is shown as a curve inFig. 3.1 is a function of the wavelength. It isempirically observed that the sensitivity of thehuman visual system is maximum in the green zone ofthe visible spectrum. The luminance sensitivityfunction gradually decays toward violet (higherfrequencies) and red (lower frequencies) from thegreen zone, as shown in the figure by the whitecurve.

In Chapter 1 on introduction of image processing, imageformation in a camera has been briefly described.Consider the image in Fig. 10.1. As a basic rule ofprojection, for a given scene point, 𝑷 , a ray from𝑷 that passes through the center of projection, 𝑶,intersects the image plane at its image point, 𝒑.This is a mapping, 𝑷 → 𝒑, of a three-dimensional(3-D) scene point to its two-dimensional (2-D) imagepoint. This rule of perspective projection isapplied for getting the image point of any scenepoint, in general. This particular geometry is thebasis of projectivegeometry in our context.

10.0.1 | Real and projective spaces

Consider a 2-D space, where a point, 𝒑, is denoted bya pair of coordinates, (𝑥, 𝑦), as shown in Fig.10.2. Since it is a cartesian product in real axis,the 2-D space is also denoted as ℝ2, and the point𝒑 belongs to the 2-D coordinate space. Followingthe coordinate conventions, these coordinates aredefined corresponding to an origin, 𝑶, and twoperpendicular axes meeting at the origin, namely,𝑥-axis and 𝑦-axis. The considered projectivespace, although defined in a 2-D space, implicitlyincludes a 3-D space behind its definition. Forexample, though all the points in an image are in a2-D plane, they are related to 3-D points of a scenewhich are lying on the ray of projection. This isthe abstraction of a 2-D projective space. Considera 3-D space, as shown in Fig. 10.2. If a ray passesthrough the origin, 𝑶, and the considered point,𝒑, 𝒑 is said to be the representative of the ray.Every point in this projected plane represents aray. In this case, the set of projection points,each representing a ray or straight line passingthrough the origin, is known as a 2-D projectivespace, ℙ2.

A document is an object which is primarily meant forhuman reading. It is not limited to only text.Besides text, which is primarily for reading, it mayalso contain figures, diagrams, photographs, tables,charts, etc. Many of these auxiliary componentsfacilitate the reading experience. Document imageprocessing involves processing of images ofdocuments. Examples of documents comprise of scannedimages of printed or handwritten pages, photographsof documents, etc. In general, images that containdifferent kinds of reading materials are consideredas images of documents. In this context, the imageof a text displayed in the environment is also anexample of document, e.g., an image of a signboard.Such kinds of texts are referred to as scenetexts.

A few examples of images of different kinds ofdocuments are shown in Fig. 16.1. The document inFig. 16.1 (a) contains printed text and graphics.Fig. 16.1 (b) is an example of an official document,which is a typical format of an official purchaseorder. Every organization usually has some standardtemplate or format of each official form used inregular administrative routines, which belongs tothis particular category of documents.

A page of magazine is shown in Fig. 16.1 (c), where thegraphics and text are overlaid in a specific layout.In Fig. 16.1 (d), an example of scene text is shown,which is a photograph of a stone tablet describing ahistorical monument. The image shown in Fig. 16.1(e) is a scanned document of a handwritten page, anexample of writing of the famous Bengali poet andNobel laureate, Rabindranath Tagore (1861–1941).

Object tracking dealswith the estimation of the trajectory of a movingobject in the image plane. The tracking takes placein a temporal sequence of images or frames in agiven input video. The task of object tracking isperformed in various contexts. Accordingly thecomputational problems are formulated with varyingset of objectives. For example, the tracking may berequired for either a single object or multipleobjects. It can either be a trajectory intwo-dimensional (2-D) space, as in the image space,or it can be a three-dimensional (3-D) trajectory,where the object is moving in a world coordinatesystem. It can either be a tracking of a set ofpoints, or it can be tracking of the entire body ofan object. The motion involved with the object caneither be a rigid body motion, or it can be adeformable body motion. The processing can either beperformed offline on recorded videos, or it may berequired to process the video in real-time or quasireal-time, while capturing the videos. On the otherhand, it could be an online tracking, which meanstracking results are obtained within a very shorttime interval of capturing of video frames. Thechoice and design of a tracking algorithm depends onthe context and application. A few examples ofapplications of tracking are shown in Fig. 8.1. TheFig. 8.1 (a) illustrates tracking of the vehiclesand pedestrians to check if they are on the correctlane. In (b), visitors of a monument in crowd arebeing tracked and in (c), cars are being tracked toensure the speed limit.

Medical imaging mostly deals with the visualization ofinternal organs, tissues, etc., using noninvasive orsemi-invasive methods. The primary motive is tounderstand any anomalies in the anatomies and theirfunctions. The signals are acquired inone-dimensional (1-D), two-dimensional (2-D),three-dimensional (3-D), or as videos, depending onthe purpose and mode of imaging. There are variousmodalities in medical imaging. The major modalitiesare as follows.

• • Projection X-ray (radiography)

• • X-ray computed tomography (CT scan)

• • Nuclear medicine images (emissiontomographies like, single photon emission computedtomography (SPECT) and positron emissiontomography (PET))

• • Magnetic resonance imaging (MRI)

• • Ultrasound

There are also several other modalities of medicalimaging. In this chapter, only the above mentionedtechniques are discussed.

14.1 | Projection X-ray imaging

The X-ray imaging technique was discovered by WilhelmConrad Rontgen (inaugural Nobel Prize, 1901) in 1895in Wurzburg, Germany. In its basic form, themechanism for generation of X-ray consists of avacuum tube with a cathode and an anodeappropriately placed as illustrated in Fig. 14.1(a). A beam of electron that emanates from thecathode hits the anode at a very high speed.Depending upon the material of the surface in theanode, there are sub-atomic interactions due to thestriking of sub-atomic particles, which releaseenergy in the form of electromagnetic waves of acertain wavelength band, called X-rays.

The subject, Computer Vision, deals with the science ofimparting to a machine or a computer the capabilityof seeing and understanding the environment as wehumans are able to do, and seeks to apply itstheories and models in various applications of ourlife and society. From the late sixties of the lastcentury, there have been efforts in analyzingdigital images captured by a scanner or a camera.Initially, it was the 2-D digital geometry in adiscrete grid of integral coordinate space whichdrew primary attention of the researchers. Inparticular, Prof. Azriel Rosenfeld (1931–2004) ofthe University of Maryland, USA, took a leading andpioneering role in developing theories of digitalpicture processing. Subsequently, the area wasstrengthened by the development and application oftheories of mathematical morphology, textureprocessing, pattern recognition techniques, etc.However, the major development in the theory ofcomputer vision, following the psycho-physiologicalmodels of human vision, happened in the seventies ofthe last century, when Prof. David Marr (1945–1980)of the Massachusetts Institute of Technology (MIT),Cambridge, USA, hypothesized three stages ofprocessing and representation of images by primalsketches consisting of edges, regions, 2.5-Dsketches of the scene, and finally 3-D models.

Over the years, theories of computer vision have beendeveloped from different areas of mathematical andphysical sciences, such as digital geometry,projective geometry, differential geometry, linearand nonlinear systems, human cognition andpsycho-visual perception, color representation andprocessing, computational learning, patternrecognition, etc. As we see, the theoreticalfoundation of the subject has been built fromdifferent domains, and it requires to learn thefundamentals across these disciplines in asystematic and organized manner in the context ofcore agenda of computer vision, which is to solveproblems related to the understanding of a 3-Dscene, static or dynamic, given visual inputs fromimaging systems.

Biometrics is the scienceand technology of uniquely identifying a person bythe physical, physiological, genomic, or behavioralcharacteristics. For example, the biometric traitsor signatures for unique characterization of aperson may be obtained from fingerprint, palm print,face, iris, retina, shape of ear, voice, signature,gait, vein in the hand, odor, handwriting, DNAsequences, etc. Some of these traits are evidentlyvisible and are often used in our socialinteractions to identify a person. But many of themmay need use of technology and computationalprocessing for extraction of biometriccharacteristic signatures from them and verifyingthem subsequently. For a unique identification of aperson, the biometric data, also referred to as thebiometric signature, should have the properties ofuniqueness and permanence. The uniqueness is thecharacteristics that uniquely identifies anindividual person and permanence implies that itshould remain unchanged throughout the life of theindividual. However, permanence in absolute sense isseldom true in practice. In view of that, it ispragmatic to use those biometric traits, which areexpected to remain mostly unaltered for asignificant period of time. During this period,there may be some marginal deviations that can belargely tolerated for a practical solution.

17.1 | Biometric system

A biometric system isprimarily designed for managing the identity of aperson. Identity management is required in almostevery sphere of social interaction and activitieslike, border control, access control to certainresources, to avail conditional facilities (food,LPG connection, etc.), financial transactions,admission to examinations, certifying thequalification and competence, etc. There may also bevarious related tasks other than identifying aperson like, determining age and gender of anindividual, establishing kinship between twopersons, etc. There are three main generic tasksthat are involved in such a system of identitymanagement.

Classification is a taskof assigning a known category or class to an object.A class is a wellstudied group of objects that is identified by theircommon properties or characteristics. For example,consider the image in Fig. 7.1, where an instance ofa region in the image is denoted by a rectangularbounding box. Here, the task is to classifydifferent regions in the image by consideringvarious patches, as illustrated by a few boundingboxes, to two classes, “human” or “nonhuman”.Likewise a few other examples of imageclassification problem are, detection of pedestriansin an image patch, recognition of a letter given atwo-dimensional (2-D) image pattern, assigning apixel of an image to its foreground or background,finding whether an image captured indoor or outdoor,etc. We may observe here the diversified nature ofclassification problems and by solving them,different types of tasks are performed. Mostly, theclassification problem falls under the supervisedlearning framework, where training samples withappropriate features and class labels are used tolearn a model that is suitable for predictingclasses of the given data. There are variousapproaches for addressing the classification problemlike probabilistic approach, distance basedapproach, discriminant analysis based approach,artificial neural network (ANN) based approach, etc.This chapter introduces four specific techniquesfrom these approaches, namely, Bayesianclassification technique (particularly, naiveBayesian classifier), 𝐾-nearest neighbor (𝐾-NN)classifier, use of linear discriminant functions,and artificial neutral network, respectively.

Computer vision is the science of facilitating amachine or a computer with the human-like capabilityof seeing and understanding the environment. It is afield of artificial intelligence (AI), which dealswith the theory, algorithmic basis, and computationfor automatic understanding of visual data acquiredfrom an environment. With the rapid advancement ofdigital and computing technology, it is possible tocapture images and videos of a scene and store thedata in the memory of a computer. Computer vision isprimarily concerned with the automatic extraction,analysis and understanding of useful informationfrom a single image, a set of images, or a videowhich is a sequence of images. It has a wide rangeof applications across the society and variousindustries, such as in autonomous vehicles, healthcare, surveillance, augmented reality, robotics,remote sensing, document processing, biometrics, andmore. Some of the key tasks of computer vision areacquisition and processing of images and videos,extracting information, and finally, derivingknowledge and description about the scene. In thisintroductory chapter, we briefly review some of thefundamental aspects of image and video processingwhich may be sufficient to follow the content of therest of the book. However, the readers may beadvised to go through first level image and videoprocessing textbooks to know more details aboutit.

1.1 Image representation

To understand how images are represented in a computer,consider an image shown in Fig 1.1. A small portionof this image, shown by a white rectangle, is zoomedto reveal enlarging details of that portion of theimage. We observe that, within this zoomed portion,although the details are better visible, the edgesappear jagged.

In this chapter, the theory and properties of singleview camera geometry are discussed. We consider theprinciple of image formation in optical cameras inthis case and apply it to relate thethree-dimensional (3-D) world with the image pointson a two-dimensional (2-D) plane.

11.1 | Pinhole camera

A mapping of a point in a 3-D coordinate space to apoint on a 2-D plane has been already discussed inthe previous chapter while explaining the canonicalconfiguration of a 2-D projective space. We relatethese concepts with respect to a pinhole camerabased imaging system. Consider a 3-D scene point, 𝑷, as shown in Fig. 11.1. The corresponding imagepoint, 𝒑′, is the point of intersection of theimage plane and the straight line from 𝑷 thatpasses through the center of the lens, 𝑂. In thesame analogy, consider the formation of an image infront of the camera center, where the correspondingimage plane is placed at the same distance as thesensor is placed behind the lens. In this case, theimages obtained on the image plane that is placed infront of the lens are of the same size as on thesensor, and there is a logical transformation ofcoordinates from point 𝒑′ to point 𝒑. Thus we maydirectly relate the scene point 𝑷 with the imagepoint 𝒑. This is a convenient way of handlingcoordinate system of image points by placing it infront of the camera in the same side of the viewingobjects.