Scalable Storage – Modern Data Warehouses



Scalable Storage

Multi-node shared storage architecture has the ability to store multiple file formats. It scales from tens of terabytes to multi-petabyte configurations and includes a comprehensive set of data services, such as replication and snapshots.

Reliability, Availability, and Serviceability (RAS):

Reliability, availability, and serviceability (RAS) are important factors in performance. Reliability means the ability of the DW system to consistently provide accurate andtimely data. Availability means the ability of the DW system to be up and running always and when needed. Serviceability is the ease with which the system can be maintained and repaired when issues arise.

RAS is essential in providing data integrity in storage systems, implementing memory controllers—including memory mirroring, enhanced error checking and correction code (ECC) scrubbing, and memory sparing—which can be highly efficient.

Implementing multiple bit errors—and with the ability to locate and correct these errors—memory scrubbers have the ability to schedule or continually read and write to memory locations, looking for failures. The engine detects, logs, and attempts to correct errors in advance. It writes an error log that allows system management to measure the time-based deterioration of dual in-line memory modules (DIMM). DIMM sparing can be used as a fail-over mechanism. Reliability can be enhanced in PCI (Payment card industry) by error detection and signal integrity.

Multiple Parallel Processing (MPP)

Having a partitioning advisor working closely with an MPP (multiple parallel processing) engine for partitioning has two objectives. Combining techniques like distributed processing, clustering, and partitioning can greatly enhance performance and efficiency. This means that multiple queries or operations can be run in parallel on the databases in the same query, if they are not dependent on the same sequence path.

Flexibility and Speed in Implementation

As part of automated development, the Data warehouse Automation (DWA) tool generates the necessary system components without manual coding. The modern data warehouse automated development service includes automated development of code, data definitions, shell scripts, documentation, testing scripts, lineage, ETL/ELT logic, scheduling, and configuration scripts. The modern data warehouse automated maintenance service includes automated operating system patches, major and minor database version upgrades, and backups.

Leave a Reply

Your email address will not be published. Required fields are marked *