Real-time Processing – Modern Data Warehouses



Real-time Processing

Advanced AIOps capabilities can favorably reduce IT administrative activities and reduce hardware admin and support costs.

Managed STaaS offerings include advanced AIOps features; consumption-based SLA metrics driving sourcing; and vendor-based asset financing and management programs.

Container-based services require a scalable persistent-data storage tier for effective usage.

Traditional data warehouses like SQL, MySQL, PostgreSQL, and Maria DB are relational databases (RDBMS) that organize data as a series of two-dimensional tables, like rows and columns, which is easy to understand. RDBMS are ACID (atomic, consistent, isolated, and durable) compliant.

Before setting expectations for NoSQL databases, which are another name for Big Data, let’s quickly recap the CAP and BASE theorems.

CAP Theorem

According to the CAP theorem (Figure 2-2), any distributed system can satisfy only two out of three properties:

Consistency: This means that everyone who reads the database should read consistent information. This means that for every read instance, the database fetches the last write consistently.

Availability: This means that the database should be always available for reading and writing. This is possible if the database has confirmation from the node about its availability. Read and write will be able to succeed when active nodes can confirm that they are live and available within reasonable response time

Partition tolerance: This means that the database system will keep functioning without interruption in the event of data loss or system failure.

Figure 2-2.  CAP Theorem

Note The partition tolerance property is a must for NoSQL databases.

Relational databases focus on the cross-section of partition tolerance and consistency. However, NoSQL is not only SQL databases. NoSQL databases focus on the cross-section between partition tolerance and availability.

The NoSQL databases work on the BASE principle, where BASE stands for the following:

•     Basic Availability: The database should be available most of the time.

•     Soft State: Temporary inconsistencies are allowed.

•     Eventual Consistency: The system will come to a consistent state after a certain period.

Leave a Reply

Your email address will not be published. Required fields are marked *