Applications

Henrique C. M. Andrade; Buğra Gedik; Deepak S. Turaga

doi:10.1017/CBO9781139058940.014

12 - Applications

from Part V - Case studies

Published online by Cambridge University Press: 05 March 2014

Henrique C. M. Andrade ,

Buğra Gedik and

Deepak S. Turaga

Show author details

Henrique C. M. Andrade: Affiliation:
J. P. Morgan
Buğra Gedik: Affiliation:
Bilkent University, Ankara
Deepak S. Turaga: Affiliation:
IBM Thomas J. Watson Research Center, New York

Book contents

Get access

Summary

Overview

In previous chapters we covered different aspects of the stream processing paradigm, including its software engineering and systems aspects, as well as its analytical foundations. In this chapter, we make use of this knowledge by taking a closer look into the design of real-world applications.

We present three applications from different business domains: a large-scale systems operations monitoring application used to manage a portion of an IT software environment, a patient monitoring application used in the context of clinical healthcare, and an application for semiconductor fabrication process control used in a manufacturing environment.

Each of these applications is described from the perspective of a system analyst who has to design the appropriate software solution. Hence, we cover the entire software engineering process, starting from gathering the requirements, designing and implementing the application components, and finalizing with the stages of optimization and refinement.

The first application, which we label Operations Monitoring (OM), is focused on analyzing log messages and events capturing the resource usage of various operational processes in a large-scale networked system shared by multiple users and processes (Section 12.2). Its aim is to continuously monitor the state of that environment, enforcing appropriate resource usage policies, as well as detecting (and potentially predicting) system outages.

This application is a front-end to different analytic applications. Its primary challenge consists of dealing with the scalability and performance requirements associated with handling large volumes of streaming data.

Type: Chapter
Information: Fundamentals of Stream Processing
Application Design, Systems, and Analytics
, pp. 441 - 484

DOI: https://doi.org/10.1017/CBO9781139058940.014 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

[1] Weka Data Mining in Java; retrieved in December 2010. http://www.cs.waikato.ac.nz/ml/weka/.

[2] Kopparapu, C. Load Balancing Servers, Firewalls, and Caches. John Wiley & Sons, Inc.; 2002.Google Scholar

[3] Streams Exchange; retrieved in May 2012. https://www.ibm.com/developer works/mydeveloperworks/groups/service/html/communityview?communityUuid=d4e7dc8d-0efb-44ff-9a82-897202a3021e.

[4] Griffin, MP, Moorman, JR. Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis. Pediatrics. 2001;107(1):97–104.CrossRef Google Scholar PubMed

[5] Vespa, P, Nuwer, M, Juhasz, C, Alexander, M, Nenov, V, Martin, N, et al.Early detection of vasospasm after acute subarachnoid hemorrhage using continuous EEG ICU monitoring. Electroencephalography and Clinical Neurophysiology. 1997;103(6):607–615.CrossRef Google Scholar PubMed

[6] Blount, M, Ebling, M, Eklund, M, James, A, McGregor, C, Percival, N, et al.Real-time analysis for intensive care: development and deployment of the Artemis analytic system. IEEE Engineering in Medicine and Biology Magazine. 2010;29(2):110–118.CrossRef Google Scholar PubMed

[7] Bonow, R, Mann, D, Zipes, D, Libby, P. Braunwald's Heart Disease: A Textbook of Cardiovascular Medicine. Elsevier Saunders Co.; 2012.Google Scholar

[8] Pan, J, Tompkins, WJ. A real-time QRS detection algorithm. IEEE Transactions on Biomedical Engineering (TBME). 1985;32(3):230–236.Google Scholar PubMed

[9] Hamilton, PS, Tompkins, WJ. Quantitative investigation of QRS detection rules using the MIT/BIH arrhythmia database. IEEE Transactions on Biomedical Engineering (TBME). 1986;33(12):1157–1165.Google Scholar PubMed

[10] CapsuleTech; retrieved in December 2010. http://www.capsuletech.com/.

[11] Electronic Industries Association – Engineering Department. Interface Between Data Terminal Equipment and Data Communication Equipment Employing Serial Binary Data Interchange. EIA; 1969.

[12] MIT PhysioBank: Arrhythmia Database; retrieved in December 2010. http://www.physionet.org/physiobank/database/mitdb/.

[13] Hastie, T, Tibshirani, R, Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer; 2009.CrossRef Google Scholar

[14] Vieira Neto, H, Nehmzow, U. Incremental PCA: an alternative approach for novelty detection. In: Proceedings of Autonomous Robotic Systems Conference (TAROS). London, UK; 2005. pp. 227–233.Google Scholar

[15] Duda, RO, Hart, PE. Pattern Classification and Scene Analysis. John Wiley & Sons, Inc.; 1973.Google Scholar

[16] IEEE. Standard for a smart transducer interface for sensors and actuators. IEEE Standards Association; 2007. IEEE 1451.5-2007.

[17] Gardner, WA. Learning characteristics of stochastic-gradient-descent algorithms: A general study, analysis, and critique. Signal Processing. 1984;6(2):113–133.CrossRef Google Scholar

[18] Hall, P, Park, BU, Samworth, RJ. Choice of neighbor order in nearest-neighbor classification. Annals of Statistics. 2008;36(5):2135–2152.CrossRef Google Scholar

[19] Aggarwal, CC. A framework for diagnosing changes in evolving data streams. In: Proceedings of the ACM International Conference on Management of Data (SIGMOD). San Diego, CA; 2003. pp. 575–586.Google Scholar

[20] Dasu, T, Krishnan, S, Venkatasubramaniam, S, Yi, K. An information-theoretic approach to detecting changes in multi-dimensional data streams. In: Proceedings of the Symposium on the Interface of Statistics, Computing Science, and Applications (Interface). Pasadena, CA; 2006.Google Scholar

[21] Kifer, D, Ben-David, S, Gehrke, J. Detecting change in data streams. In: Proceedings of the International Conference on Very Large Databases (VLDB). Toronto, Canada; 2004. pp. 180–191.Google Scholar

[22] Dongarra, J, Duff, I, Sorensen, D, van der Vorst, H, editors. Numerical Linear Algebra for High Performance Computers. Society for Industrial and Applied Mathematics; 1998.CrossRef

[23] Adobe. Using Adobe Flex 4.6. Adobe Systems; 2011.

[24] Apache Flex; retrieved in September 2012. http://http://incubator.apache.org/flex/.