In continuation of our Data Science series of basic concepts and knowledge, today we will discuss the differences between Database, Data warehouse, and Data lake.
Database This is undoubtedly a familiar concept for IT professionals.
A database is a collection of organized data that is typically accessed from a computer system or stored in file format within a database management system. Databases are used to store, search, and report on structured data from a single source.
There are several popular database models today:
File-based databases: The most common type of database where data is stored in file format. They are easily accessible and simple to organize. Relational databases: Different data sources are merged and stored in the same table. These data must be related to each other. Examples of databases that support relational databases include Oracle, MS SQL Server, and MySQL. Object-oriented databases: A pure data table that adds additional object-oriented fields such as object behavior. These objects are clearly classified, each level is called a data class. Groups of objects are collected in the same table and represented by a data row. Semi-structured databases: Can store many different types of data and are often stored in XML format. Data and object descriptions are presented in tag tags. They are easily expandable and convenient, allowing users to easily access the necessary information.
Data Warehouse Initially defined as a dataset used to support decision-making processes, the Data Warehouse has evolved into an information environment that has the following functions:
Provide a comprehensive view of the business Provide complete current and historical information about the business, and be ready to be exploited to support strategic decision-making Ensure consistent information Flexible and interactive information, which means that users can obtain different information about the same object with multiple operations, rather than returning a static list.
- Characteristics of Data Warehouse:
Subject-oriented: Data in a data warehouse is organized and arranged by a specific topic. Integrated: The ability to aggregate data from multiple sources into a data warehouse. Time-variant: Time labeling makes it easy to compare data to each other for analysis.
Data Lake
A data lake is a system that stores raw and unstructured data in its native format. Unlike a data warehouse, a data lake does not require structured data to be organized in advance, nor does it have a pre-defined schema. This means that data can be stored and processed more quickly and easily. Data lakes are typically used to store data that may be valuable in the future, but whose use cases and applications have not yet been defined.
In conclusion, databases, data warehouses, and data lakes all have their own characteristics and are suitable for different types of data storage and processing. It's important to understand the differences between them to determine which one is best for your organization's needs.