The easy access to high-performance computing resources in cloud computing has not only made the process-intensive activities smarter but the data-intensive computing activities also have taken center stage. The nature of data have radically changed with this revolutionary utility service; hence their processing and storage requirements vary. Large data-sets are generated and produced everyday are sent for processing in the high-performance computing environments.
Like the traditional storages, users can store and access multimedia files of various formats like text, image, audio and video in cloud also, but the storage requirements have been altered for efficient processing of the large data-sets which are produced in cloud every hour. The traditional enterprise level files and data storage systems were not sufficient to satisfy all of the data-intensive and high-performance computing requirements.
Efficient file handling to support parallel and distributed operations of large data-sets needed an entirely new file system format. Hence, researchers and computing vendors have come up with suitable storage solutions to achieve optimal performance in cloud like high-performance environment. This chapter focuses on all of these advancements made to fulfill those requirements.
Cloud computing promises high-performance. Hence, the file system and storage to support high-performance data processing are critical requirements of cloud environment.
REQUIREMENTS OF DATA-INTENSIVE COMPUTING
Data-intensive computing presents a challenge to computing systems in terms of delivering high-performance. Large volume complex data-sets cannot be processed centrally in a single node and require partitioning and distribution over multiple processing nodes. Thus, data-intensive computing is I/O-bound and requires rapid movements of data in large numbers. This requires appropriate management of data in transaction. Data modelling, partitioning, node assignment and accumulation are some of the critical parts of this computing. Consumers’ requirements and technical aspects related to the storage facility in high-performance computing environment are different from the traditional storage system in many ways. Traditional enterprise storage systems are no more sufficient to tackle those issues.
Scalability and high-performance distributed data processing are complementary to each other. Distribution of large data-set among as-many-nodes as required for processing promotes scalability of the application. Suitable file systems are required to support this distribution and scaling proficiently. It has been observed that data-intensive computing often involves process-intensive computing too. Complex data-sets present challenges before the computing system.
Review the options below to login to check your access.
Log in with your Cambridge Higher Education account to check access.
If you believe you should have access to this content, please contact your institutional librarian or consult our FAQ page for further information about accessing our content.