Several major technologies of cloud storage systems

Cloud storage system has the following characteristics: data security, superior scalability, according to the use of charges, can automatically switch faults across different applications, easy to manage. Cloud storage is mainly used in the four major areas of backup, archiving, distribution, and sharing collaboration.

Cloud storage is a new concept extended and developed in the concept of cloud computing. It refers to the aggregation of a large number of different types of storage devices in the network through application software through cluster applications, grid technologies, or distributed file systems. A system that works together to provide external data storage and business access capabilities. When the core of cloud computing system operation and processing is the storage and management of large amounts of data, a large number of storage devices need to be configured in the cloud computing system. Then the cloud computing system is transformed into a cloud storage system, so cloud storage is a data storage system. And management-centric cloud computing systems.

Compared with cloud computing systems, cloud storage can be considered as a cloud computing system with a large-capacity storage space. Cloud storage system has the following characteristics: data security, superior scalability, according to the use of charges, can automatically switch faults across different applications, easy to manage. Cloud storage is mainly used in the four major areas of backup, archiving, distribution, and sharing collaboration.

Cloud storage system is a collection of multi-device, multi-application and multi-service collaborative work. Its implementation must be based on the development of multiple technologies. According to the characteristics of cloud storage and its application areas, the main cloud storage technologies involve storage virtualization, distributed file systems, cluster storage, centralized storage management, heterogeneous platform collaboration, automatic hierarchical storage, and of course, deduplication. , data compression and other technologies.

First, Storage Virtualization The most common understanding of Storage Virtualization is the abstraction of storage hardware resources. Integrate one (or more) target services or functions with other additional functions to provide a unified and comprehensive functional service. Typical virtualization includes the following situations: masking the complexity of the system, adding or integrating new functions, simulating, integrating, or decomposing existing service functions. Virtualization is applied to one or more entities, and these entities are used to provide storage resources or services.

Storage virtualization is a technology that runs through the IT environment to simplify the underlying infrastructure that might otherwise be relatively complex. The idea of ​​storage virtualization is to separate the logical image of the resource from the physical storage, thereby providing the system and administrator with a simplified and seamless virtual view of the resource.

For users, virtualized storage resources are like a huge "storage pool". Users do not see specific disks or tapes, nor do they need to know which path leads to which specific storage device. .

Second, the distributed file system Hadoop Distributed File System (HDFS) is a distributed file system designed for use on ordinary hardware devices. It has many similarities with existing distributed file systems, but it is very different from these file systems. HDFS is highly fault-tolerant and is designed to be deployed on inexpensive hardware. HDFS provides high throughput for application data and is suitable for those large data set applications. HDFS opens up some of the POSIX required interfaces to allow streaming access to file system data.

HDFS is master/slave. A cluster has a name node, which is the master control server, responsible for managing the file system's namespace and coordinating client access to files. There are also a bunch of data nodes, usually one deployed on a physical node, responsible for the storage management at the physical nodes where they are located. HDFS opens the file system's namespace so that user data is stored in files. Internally, a file is divided into one or more data blocks, which are stored in a set of data nodes. The name node performs name space operations on the file system, such as opening, closing, renaming files or directories, and also determines the mapping of data blocks from data nodes. The data node is responsible for providing customer's read and write requests. The data node also performs the creation, deletion, and copying of data blocks in accordance with the instructions of the name node.