📔 Database Kernel develop RoadMap

Author is a database amateur, so the content is not authoritative, welcome to point out my error.

Since the database course is recommended, only good articles, papers or blogs are listed starting from part 3.

List of Knowledge

• List of Knowledge
• 1. Description
• 2. Recommended Basic Course and Books
  • Pre-Skill
  • 2.1 Computer Organization and Design
    • 2.1.1 Courses
    • 2.1.2 Books
  • 2.2 Computer Operating System
    • 2.2.1 Courses
    • 2.2.2 Books
  • 2.3 Computer Network
    • 2.3.1 Courses
    • 2.3.2 Books
  • 2.4 Data Structure and Algorithmn
    • 2.4.1 Courses
    • 2.4.2 Books
  • 2.5 Algorithms Design and Analysis
    • 2.5.1 Courses
    • 2.5.2 Books
  • 2.6 Database
    • 2.6.1 Course
    • 2.6.2 Books
    • 2.6.3 Papers
  • 2.7 Distributed System
    • 2.7.1 Courses
    • 2.7.2 Books
    • 2.7.3 Papers
    • 2.7.4 Blogs
• 3. SQL & Relational Algebra
• 4. DDL & DML
• 5. Relational Model
• 6. Storage management
  • 6.1 Buffer Pool
  • 6.2 Index
  • 6.3 B+Tree
  • 6.4 B-Tree
  • 6.5 Hash table
  • 6.6 LSM-Tree
  • 6.7 Bw-Tree
  • 6.8 Storage engine
• 7. Query Processing
• 8. SQL Parser
  • 8.1 Syntax Check
  • 8.2 Semantic Check
  • 8.3 Shared Pool Check
  • 8.4 Actions
• 9. SQL Executor
• 9. SQL Optimization
  • 9.1 SQL Query Optimization
  • 9.2 Indexes Optimization
  • 9.3 Table Optimization
  • 9.4 Buffer Cache Optimization
  • 9.5 Storage Optimization
  • 9.7 table Structure Optimization
    • 9.7.1 data reduction
    • 9.7.2 data partition
  • 9.8 Optimizer
  • 9.9 SQL调优博客List
• 10. Transaction management
• Lock manager
• 11. Network
• 12. Serialization
• 13. Concurrency Control
• 14. Crash Recovery management
• 15. NoSQL
• 16. NewSQL
• 17. Distributed & Paralleled
• 15. OLAP、OLTP、HTAP
  • 15.1 OLAP
  • 15.2 OLTP
  • 15.3 HTAP
  • 15.4 HSAP
• 16. Graph Database
  • 16.1 Courses
  • 16.2 Books
  • 16.3 Papers
  • 16.4 Blogs
• 16. Project Source Code Analysis
• 17. Mini-Project Labs
• 18. AI4DB and DB4AI (frontier tech)
• Database Retrospective

1. Description

This repo will not include knowledge about basic programming language. it mainly focus on direction of database kernal development.

The tutorial will include content:

• Courses
• Blogs
• Papers
• Activities
• Talks

Ok, The repo include two language content：

• English Content
• Chinese Content.

If you have any good materials, welcome to submit a pull request for this repo.

Finally, I would like to thank the professor who provided the Course resource, papper resource Etc. Also thanks the blogger who provided the study notes. Thanks to all content resource providers.

2. Recommended Basic Course and Books

I will not update course notes and labs.

Pre-Skill

• MIT The Missing Semester of Your CS Education
• C/C++
• Go
• Rust

You only need to know a programming language.

2.1 Computer Organization and Design

2.1.1 Courses

2.1.2 Books

• Reference Course Resources

2.2 Computer Operating System

2.2.1 Courses

2.2.2 Books

• Reference Course Resources

2.3 Computer Network

2.3.1 Courses

2.3.2 Books

• Reference Course Resources

2.4 Data Structure and Algorithmn

2.4.1 Courses

2.4.2 Books

• Reference Course Resources

2.5 Algorithms Design and Analysis

2.5.1 Courses

2.5.2 Books

• Reference Course Resources

2.6 Database

2.6.1 Course

2.6.2 Books

• fundamentals of database systems 7th edition solutions, by Ramez Elmasri, Shamkant B. Navathe

• Database System Concepts Seventh Edition, by Silberschatz, Korth and Sudarshan 中文版：《数据库系统概念》

• Database Management Systems, by Ramakrishnan and Gehrke 中文版：《数据库管理系统原理与设计》

• Database Internals A Deep-Dive into How Distributed Data System Work, by Alex Petrov 中文版：《数据库系统内幕》

• Designing Data-Intensive Applications, by Martin Kleppmann

• Database Design and Implementation, by Edward Sciore

• Database System Implementation， by Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom 中文版：《数据库系统实现》

  • This book has been replaced by Database Systems: The Complete Book

• Transaction Processing Concepts and Techniques by Jim Gray

• Principles of Distributed Database Systems, by M. Tamer Özsu and Patrick Valduriez 中文版：《分布式数据库系统原理》

• nosql distilled a brief guide to the emerging world of polyglot persistence, by Pramod J. Sadalage and Martin Fowler 中文版：《NoSQL精髓》

• High Performance MySQL Third Edition, by Baron Schwartz, Peter Zaitsev, Vadim Tkachenko 中文版：《高性能MySQL》

• MySQL High Availability: Tools for Building Robust Data Centers, by CharlesBell,MatsKindahl,LarsThalmann 中文版：《高可用MySQL:构建健壮的数据中心》

2.6.3 Papers

• CS 15-721 Topics Papers

• Stanford CS 345 Winter2014 Topic Paper

• Paper 2017 : How to Build a Non-Volatile Memory Database Management System

• Aticle : Main Memory Database Systems

• Paper thesis 2018 : The Design and Implementation of a Non-Volatile Memory Database Management System

• harvard Aticle : The Design and Implementation of Modern Column-Oriented Database Systems

• In-Memory Database Systems - A Paradigm Shift

2.7 Distributed System

This Section is for distributed database.

• Notes on Distributed Systems for Young Bloods

2.7.1 Courses

2.7.2 Books

• Distributed Systems 3rd edition (2017)

• Distributed systems for fun and profit

• Data-Intensive Text Processing with MapReduce

• Distributed Systems: Principles and Paradigms

• Principles of Computer System Design

• designing data-intensive applications , DDIA

• Introduction to Distributed Algorithms

• Distributed Storage Systems: Principles, Algorithms and Designs

• Distributed Systems: Concepts and Design

2.7.3 Papers

• Reference :

  • http://reiddraper.github.io/distreader/
  • MIT6.824 : http://nil.csail.mit.edu/6.824/2015/schedule.html
  • http://henryr.github.io/distributed-systems-readings/

• Causality

  • Paper 1978 : Time, Clocks, and the Ordering of Events in a Distributed Syste
  • Paper 2008 : Interval Tree Clocks: A Logical Clock for Dynamic Systems , ITC2008

• Consistency

  • Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services

• Consensus

  • consensus-bridging-theory-and-practice
  • Raft Refloated: Do We Have Consensus?
  • The Part Time Parliament
  • Memory Coherence in Shared Virtual Memory Systems
  • TreadMarks: Distributed Shared Memory on Standard WorkStations and Operating Systems
  • Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System
  • Resolving File Conflicts in the Ficus File System
  • Consistency Analysis in Bloom: a CALM and Collected Approach
  • Paper 2009 : Object Storage on CRAQ High-throughput chain replication for read-mostly workloads , USENIX2009
  • Distributed SQLite: a replicated SQLite service powered by RAFT

• Data Stores

  • Simplified Harp File System
  • Google File System 2.0: A Modern Design and Implementation
  • Replication in the Harp File System
  • Simplified GFS
  • Chunky: a distributed GFS-based file store
  • Append-only Datastore
  • Key-Value Store Using Chain Replication
  • ACID Compliant Distributed Key-Value Store
  • RAFT based Key-Value Store with Transaction Support

• General

  • Paper 2010 : The Declarative Imperative: Experiences and Conjectures in Distributed Logic , SIGMOD2010
  • Paper 2004 : MapReduce: Simplified Data Processing on Large Clusters , OSDI2004
  • Paper : Availability in Globally Distributed Storage Systems

• Fault Tolerance

  • Remus: High Availability via Asynchronous Virtual Machine Replication
  • Paxos Made Simple , ACM SIGACT News 2001
  • In Search of an Understandable Consensus Algorithm
  • Harvest, Yield, and Scalable Tolerant Systems
  • Paxos Made Moderately Complex
  • Distributed Multi-Agent Consensus for Fault Tolerant Decision Making

• Atomicity

  • The Akamai Network: A Platform for High-Performance Internet Applications
  • Argus : DISTRIBUTED PROGRAMMING IN ARGUS
  • Thor : Efficient Optimistic Concurrency Control Using Loosely Synchronized Clocks

• Application

  • Paper 2003 : The Google File System , SOSP2003
  • Paper 2003 : Bigtable: A Distributed Storage System for Structured Data , OSDI2006
  • Paper 2007 : Dynamo: Amazon’s Highly Available Key-value Store , SOSP2007
  • Peer-to-Peer Note Taking App
  • factoryOS: A Distributed and Self-Organizing Planning System in a Supply-Chain Context
  • Megastore: Providing Scalable, Highly Available Storage for Interactive Services , CIDR 2011
  • PNUTS: Yahoo!’s Hosted Data Serving Platform , VLDB 2008

2.7.4 Blogs

• Oracle7 Server Concepts Manual : Distributed Databases

• University of Cambridge : Distributed Systems

• Replication, atomicity and order in distributed systems

• Distributed Systems and the End of the API

• MIT Theory of Distributed Systems (TDS)

• 分布式系统学习资料汇总 , Blogger : 木鸟杂记

• 分布式系统(Distributed System)资料 , Github Blogger : ty4z2008

• Notes on Paxos

• Understanding Paxos

• Paxos 学习笔记1 - Basic Paxos

• Paxos 学习笔记2 - Multi-Paxos

• Raft 学习笔记1 - 领导人选举和日志复制

• Raft协议详解

3. SQL & Relational Algebra

• SQL 知识点：Book《Database System Conceptsm》Chapter3、Chapter4、Chapter5

• Relational Algebra 知识点：Book《Database System Conceptsm》Chapter6

• w3cSchools SQL: https://www.w3schools.com/sql/

• Introduction of Relational Algebra in DBMS

• Relational Algebra in DBMS: Operations with Examples

• slide : Relational Algebra and SQL

• SQL深入理解|关系代数、简单查询、连接

• SQL 与关系代数

4. DDL & DML

• DDL : https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language

• DML : https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax

• Difference Between DDL and DML in DBMS

• MySQL · 源码分析 · 原子DDL的实现过程

• MySQL · 源码分析 · 8.0 原子DDL的实现过程【续】

• MySQL · 源码分析 · DDL log与原子DDL的实现

• PolarDB · 优化改进 · DDL的优化和演进

• PolarDB · 功能特性 · 非阻塞DDL

• MySQL · 源码分析 · 8.0 · DDL的那些事

• X-Engine · 引擎特性 · 并行DDL

5. Relational Model

• WikiPedia : Relational model

• Relational Data Model in DBMS | Database Concepts & Example

• What is ER Modeling? Learn with Example

• ER (Entity Relationship) Diagram in DBMS

• Relational Model in DBMS

• 数据库系统原理学习笔记（二）-数据库设计和ER图

• Understanding Relational Databases

• Understanding the Enhanced ER Model Simplified 101

• Data Model Relationships 101: Simplified Guide for Beginners

• 9 ER Model Tools to use in 2023: A Comprehensive List

6. Storage management

CS 15-445 课程 Lecture03、Lecture04

• Managing Database Storage Structures

• Some study on database storage internals

• Storage 101 Series

  • Storage 101: Understanding the Hard-Disk Drive

  • Storage 101: Understanding the NAND Flash Solid State Drive

  • Storage 101: RAID

• RAID

• RAID and Its Impact on your SQL Performance

• Importance of RAID in Databases

• Microsoft Research Paper : Faster: A Concurrent Key-Value Store with In-Place Updates, SIGMOD2018

• How Lock-free Data Structures Perform in Dynamic Environments: Models and Analyses

6.1 Buffer Pool

• CS 15-445 课程 Lecture05

• MySQL · 性能优化· InnoDB buffer pool flush策略漫谈

• MySQL · 引擎特性 · InnoDB Buffer Pool 浅析

• MySQL · 引擎特性 · Buffer Pool 漫谈

• MySQL · 引擎特性 · InnoDB Buffer Pool

• MySQL · 引擎特性 · InnoDB Buffer Page 生命周期

• InnoDB 的 Buffer Pool 分析

6.2 Index

• wikiPedia : Database index

• Google Cloud docs : Indexes

• Database Indexes Explained

• An in-depth look at Database Indexing

• Database Index: An Introduction for Beginners

• Indexing in DBMS: What is, Types of Indexes with EXAMPLES

• Database · 最佳实践 · 内存索引指南

• MongoDB · 特性分析 · 索引原理

• MongoDB · 引擎特性 · MongoDB索引原理

• Sparse Indexes（稀疏索引）

• MySQL · 引擎特性 · 二级索引分析

• MySQL · 源码阅读 · 创建二级索引

6.3 B+Tree

• wikiPedia : B+ tree

• CS 186 Spring notes pdf : B+ tree

• University of Utah sliede : Database Systems Index: B+ Tree

• implement a B+ Tree index file

• Paper 2004 : Query and Update Efficient B-Tree Based Indexing of Moving Object

• MySQL · 引擎特性 · B+树并发控制机制的前世今生

• Innodb 中的 Btree 实现 (一) · 引言 & insert 篇

• POLARDB · 理论基础 · 敢问路在何方 — 论B+树索引的演进方向（上）

• POLARDB · 性能优化 · 敢问路在何方 — 论B+树索引的演进方向（中）

• MySQL · 源码分析 · innodb 空间索引实现

• MySQL · 引擎特性 · 手动分析InnoDB B+Tree结构

6.4 B-Tree

• 课程：《Let's Build a Simple Database》

  • Part 7 - Introduction to the B-Tree
  • Part 8 - B-Tree Leaf Node Format

• B- Tree Datastructure

• B- Trees : Software Design Using C++

• Paper 2010 , Efficient B-tree Based Indexing for Cloud Data Processing, VLDB, National University of Singapore & IBM Watson Research Center

• Database · 理论基础 · 高性能B-tree索引

• 要懂Greenplum索引，心里得有B树！

6.5 Hash table

• wikipedia : Hash table

• CMU CS15-445 slide : Hash table slide

• GAtech.edu slide : Hash table & Extendible Hash & Linear Hash

• Blog : An Introduction to B-Tree and Hash Indexes in PostgreSQL

• Extendible Hashing (Dynamic approach to DBMS)

• Extendible hashing for COSC 311

6.6 LSM-Tree

• What is a LSM Tree?

• Log Structured Merge Trees

• slide : Log Structured Merge Tree, thanks to Pinglei Guo

• Paper : The Log-Structured Merge-Tree (LSM-Tree)

• AlibabaCloud Community Blog : Starting from Zero: Build an LSM Database with 500 Lines of Code

• B-Tree vs Log-Structured Merge-Tree

6.7 Bw-Tree

• Paper : The Bw-Tree: A B-tree for New Hardware Platforms , Microsoft Research

• Lock-free data structures. Inside. Memory management schemes

• Blog : Bw-Tree技术解读

• Blog : 微软提出的无锁 B 族树 —— Bw-Tree , 木鸟杂记

6.8 Storage engine

• The Beginner’s Guide to MySQL Storage Engines

• MySQL Storage Engines

• MySQL 8.0 docs : Chapter 15 The InnoDB Storage Engine

• MySQL 8.0 docs : Chapter 16 Alternative Storage Engines

• Hybrid storage engine for geospatial data using NoSQL and SQL paradigms

• Oracle docs : Chapter 13. Storage Engines

• Paper 2021 : Designing a persistent-memory-native storage engine for SQL database systems , , IEEE NVMSA 2021

  • Paper : https://nvmsa2021.github.io/paper/NVMSA_2021_Session_5-2_paper.pdf
  • Slide : https://nvmsa2021.github.io/slides/NVMSA_2021_Session_5-2_slides.pdf

• MongoDB · 特性分析 · MMAPv1 存储引擎原理

7. Query Processing

• Distributed Query Processing (DQP)

• Database SQL Tuning Guide : 3 SQL Processing

8. SQL Parser

• SQL Parser API

• Examples of SQL parsing

• SQL语言解析器的实现

• Parser 解析重写 SQL

• openGauss内核分析（三)：SQL解析

8.1 Syntax Check

• SQL Syntax Checker Tools

8.2 Semantic Check

• Paper : Semantic Parsing with Syntax- and Table-Aware SQL Generation

• Awesome Sequence to SQL and Semantic Parsing

8.3 Shared Pool Check

• Shared Pool Management

• Blog : Oracle Identifying Shared Pool Contention

• 12.2 DBMS_SHARED_POOL: Pinning Objects

8.4 Actions

• Surveying SQL parser libraries in a few high-level languages

• SQL解析在美团的应用

• TiDB 源码阅读系列文章（五）TiDB SQL Parser 的实现

• MYSQL · 新特性 · MySQL 8.0对Parser所做的改进

• MySQL · 源码分析 · 词法分析及其性能优化

9. SQL Executor

• SQL Executor, How does it work?

• Paper : Adaptive Execution of Compiled Queries

• MySQL · 内核特性 · 8.0 新的火山模型执行器

9. SQL Optimization

• 15 Best Practices for SQL Optimization

• Tips for SQL Database Optimization

• Design and Practice of Self-Developed SQL Parser

• Paper : A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese

9.1 SQL Query Optimization

• Top 10 SQL Query Optimization Tips to Improve Database Performance, by Avishek Singh

• SQL Query Optimization: How to Tune Performance of SQL Queries

• Oracle docs : 5 Query Optimization

• The SQLite Query Optimizer Overview

• How to Optimize SQL Queries: Helpful Tips and Techniques

• 基于代价的慢查询优化建议

9.2 Indexes Optimization

• Google Cloud docs : Optimizing Indexes

• How to use Indexing for SQL Query Optimization

• Database Optimization Techniques #1: Indexing

• Understanding The Techniques Of Database Indexing

• Optimizing SQL Server index strategies

• Indexes - Optimize Queries

• Examples: Using Indexes for Query Optimization

• Optimize index maintenance to improve query performance and reduce resource consumption

• How to create and optimize SQL Server indexes for better performance

• Tutorial on MySQL Database Optimization using Indexes

• MySQL索引原理及慢查询优化

• SQL优化 · 经典案例 · 索引篇

9.3 Table Optimization

• How To Optimize MySQL Tables

• MySQL Optimize Table: How to Keep Your Database Running Smoothly

• MySQL docs : 8.5 Optimizing for InnoDB Table

9.4 Buffer Cache Optimization

• Buffer cache: What is it and how does it impact database performance?

• EXPLAIN (ANALYZE) needs BUFFERS to improve the Postgres query optimization process

9.5 Storage Optimization

• Introduction to Memory-Optimized Tables

• Best Practices for MySQL Database Storage Optimization

• AWS Storage Blog : Storage for I/O-intensive SQL Server using Amazon EBS io2 Block Express

• Data Storage and Optimization

9.7 table Structure Optimization

9.7.1 data reduction

• MySQL docs : 15.9.1.5 How Compression Works for InnoDB Tables

9.7.2 data partition

• When and how to use the SQL PARTITION BY clause

• SQL Database, Table and Data Partitioning: When and How to Do It

9.8 Optimizer

Reference : https://zhuanlan.zhihu.com/p/363997416 , thanks henry liang

• Paper : Orca: A Modular Query Optimizer Architecture for Big Data , SIGMOD 2014

• Paper : An Overview of Cost-based Optimization of Queries with Aggregates

• Paper : Optimizer plan change management: improved stability and performance in Oracle 11g , VLDB

• Paper : Optimizing Queries over Partitioned Tables in MPP Systems , SIGMOD

• Paper : Optimization of Common Table Expressions in MPP Database Systems , VLDB

9.9 SQL调优博客List

• SQL调优实战总结

• OceanBase SQL 调试优指南 , Alibaba Ant Group

• PolarDB-X SQL 调优指南 , Alibaba Cloud

• TiDB SQL性能调优 , PingCAP

• SQL调优常用方法

• MySQL 性能调优

• 了解数据库性能优化 ， thanks to TiDB

• 浅谈 数据库应用与 SQL 调优的原理

• 一文搞定MySQL性能调优

• TiDB SQL调优实战——索引问题

• MySQL 调优笔记 ， Github Blogger : wardseptember

• 10 essential MySQL performance tuning tips

• 101 MySQL tuning and optimization tips , Alibaba Cloud

10. Transaction management

• Google Cloud docs : Transactions

• Paper 2006, Cost-based query transformation in Oracle, VLDB

• 分布式事务，理论与实践

• MySQL · 引擎特性 · InnoDB 事务系统

• MySQL · 引擎特性 · InnoDB 事务子系统介绍

• Database · 原理介绍 · 数据库的事务与复制

• MongoDB · 引擎特性 · 事务实现解析

• Innodb中的事务隔离级别和锁的关系

• Database · 理论基础 · 数据库事务隔离发展历史

• Database · 原理介绍 · Google Percolator 分布式事务实现原理解读

• 第 12 章 事务和并发

• Blog : 使用数据库事务

• Blog : 11. 数据库事务

• InnoDB 事务分析-MVCC

• InnoDB 事务分析-Undo Log

Lock manager

• MySQL · 引擎特性 · 8.0 Lock Manager

• MySQL · 引擎分析 · InnoDB行锁分析

• MySQL · 引擎特性 · InnoDB index lock前世今生

• MySQL · 最佳实践 · How to read the lock information from debugger

• PostgreSQL · 内核特性 · 死锁检测与解决

• MySQL · 引擎特性 · InnoDB 事务锁系统简介

• MySQL · 引擎特性 · Innodb 锁子系统浅析

• MySQL · 答疑解惑 · MySQL 锁问题最佳实践

• MySQL · 源码分析 · InnoDB读写锁实现分析

• InnoDB 事务 sharded 锁系统优化

• MySQL · myrocks · 事务锁分析

• MySQL · 源码分析 · 事务锁调度分析

• MySQL · 引擎分析 · InnoDB行锁分析

• DataBase · 理论基础 · B+树数据库加锁历史

11. Network

• MySQL · 源码分析 · 网络通信模块浅析

• MySQL · 引擎特性 · 网络模块优化

• MongoDB · 特性分析 · 网络性能优化

12. Serialization

• 可序列化的初学者指南

• AWS Redshift Blog : 可序列化的隔离

• 从SPI机制学习数据库驱动加载过程到SnakeYaml反序列化分析

• MySQL · 源码分析 · Tokudb序列化和反序列化过程

• 美团技术团队博客 : 序列化和反序列化

13. Concurrency Control

• MySQL · 引擎特性 · B+树并发控制机制的前世今生

• 浅析数据库并发控制机制

• Database · 理论基础 · B-tree 物理结构的并发控制

• PolarDB · 引擎特性 · B-tree 并发控制优化

• MySQL · 最佳实践 · RDS MySQL 8.0 语句级并发控制

• PgSQL· 引擎特性 · 多版本并发控制介绍及实例分析

• 数据库系统 · 事务并发控制 · Two-phase Lock Protocol

• How does MVCC (Multi-Version Concurrency Control) work

• Efficient Locking for Concurrent Operations on B-Trees

• Database · 理论基础 · B link Tree

• A Blink Tree method and latch protocol for synchronous node deletion in a high concurrency environment

• 周刊（第23期）：图解Blink-Tree：B+Tree的一种并发优化结构和算法

• Mass Tree Paper : Cache Craftiness for Fast Multicore Key-Value Storage

• Database · 理论基础 · Mass Tree

• Palm Tree Paper : PALM: Parallel Architecture-Friendly Latch-Free Modifications to B+ Trees on Many-Core Processors

• Database · 理论基础 · Palm Tree

• ARTree Paper : The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases

• Database · 理论基础 · Multi-ART

14. Crash Recovery management

• MySQL · 引擎特性 · InnoDB 崩溃恢复过程

• PgSQL · 特性分析 · checkpoint机制浅析

• PgSQL · 特性分析 · 数据库崩溃恢复（上）

• PgSQL · 特性分析 · 数据库崩溃恢复（下）

• MySQL · TokuDB · 日志子系统和崩溃恢复过程

• MySQL · myrocks · myrocks之备份恢复

• POLARDB · 理论基础 · 数据库故障恢复机制的前世今生

• POLARDB · 引擎特性 · PolarDB备份与恢复介绍

• B+树数据库故障恢复概述

• MySQL · 5.7改进 · Recovery改进

• MySQL · 源码分析 · binlog crash recovery

15. NoSQL

• NoSQL Tutorial: What is, Types of NoSQL Databases & Example

16. NewSQL

• Database · 发展前沿 · NewSQL数据库概述
• 我们是怎样打造一款分布式数据库的？
• 如何编写一个分布式数据库？ , PingCAP CEO刘奇
• Understand the Differences Between NewSQL and Distributed SQL

17. Distributed & Paralleled

• Experiences with a Distributed, Scalable, Methodological File System: AnalogicFS

• Monarch: Google’s Planet-Scale In-Memory Time Series Database

• Paper 2012 : Spanner: Google's Globally-Distributed Database , OSDI

15. OLAP、OLTP、HTAP

15.1 OLAP

• WikiPedia : OnLine Analytical Processing

• AWS Blog : What Is Online Analytical Processing?

• Azure Blog : Online analytical processing (OLAP)

• An Overview of Data Warehousing and OLAP Technology

15.2 OLTP

• wikiPedia : Online transaction processing

• What is OLTP? Definition, Architecture, Example

• Azure Blog : Online transaction processing (OLTP)

• LAP vs. OLTP: Understanding 13 Crucial Differences

15.3 HTAP

• Hybrid transactional/analytical processing

• Tsinghua University databaseGroup Sigmod2022 slide : HTAP database : A Tutorial

• What is HTAP?

• A Common Database Approach for OLTP and OLAP Using an In-Memory Column DataBase

• PingCAP : How We Build an HTAP Database That Simplifies Your Data Platform

• F1 Lightning: HTAP as a Service论文阅读笔记

• 【技术硬核】从Google F1 看HTAP数据库的诞生

• PolarDB HTAP的功能特性和关键技术

• 软硬协同的HTAP数据库系统

• StoneDB 文章

  • 什么是真正的HTAP？（一）背景篇
  • 什么是真正的 HTAP ？（二）挑战篇

• AlibabaCloud Community Blog : : 400x Faster HTAP Real-time Data Analysis with PolarDB

• GreenPlum database Blog : World Class Open Source Distributed HTAP Database Based On PostgreSQL

• Paper 2020 : TiDB: A Raft-based HTAP Database, VLDB

• PolarDB for PostgreSQL HTAP 架构详解

• Paper 2022 : Kernel-Assisted Copy-on-Write Snapshots for Main-Memory HTAP Databases

• DataBase · 引擎特性 · OLAP/HTAP列式存储引擎概述

• DataBase · 理论基础 · HTAP列存引擎探秘

• PolarDB MySQL·HTAP·浅析IMCI的列存数据压缩

• MySQL · HTAP · 分析型执行引擎

• HybridDB · 最佳实践 · OLAP和OLTP一体化打造

• MySQL · HTAP · 分析型执行引擎

15.4 HSAP

• Paper : Alibaba Hologres: A Cloud-Native Service for Hybrid Serving/Analytical Processing

  • VLDB Link : http://www.vldb.org/pvldb/vol13/p3272-jiang.pdf
  • 《Alibaba Hologres: A Cloud-Native Service for Hybrid Serving/Analytical Processing 》论文导读

• AlibabaCloud Community Blog : What Is the Next Stop for Big Data? Hybrid Serving/Analytical Processing (HSAP)

16. Graph Database

16.1 Courses

• YouTube Video : Introduction to Graph Databases Series

16.2 Books

• Graph Database : New Oppotunities For Connected Data , Ian Robinson, Jim Webber & Emil Eifrem

• Graph Databases For Beginners , Merkl Sasaki, Joy Chao & Rachel Howard

• Graph-Databases-For-Dummies , Dr. Jim Webber & Rik Van Bruggen

• The Definitive Guide to Graph Databases for the RDBMS Developer , Michael Hunger, Ryan Boyd & William Lyon

16.3 Papers

• Paper 2022 : ByteGraph: A High-Performance Distributed Graph Database in ByteDance , VLDB

16.4 Blogs

• ByteGraph: A Graph Database for TikTok

• 字节跳动自研万亿级图数据库 & 图计算实践

• 美团技术团队 Blog : 美团图数据库平台建设及业务实践

• Graph Databases for Beginners: Why Graph Technology Is the Future

• Graph Databases: How They Work, When to Use Them & the Advantages They Offer

• Developing a Small-Scale Graph Database: A Ten Step Learning Guide for Beginners

• Graph Database Tutorial With Neo4j

• Getting started with Graph database using Neo4j

• 白皮书 : 图数据库技术十大案例

16. Project Source Code Analysis

Just collect, some databases have not been read yet. Thanks to all Authors.

taobao MySQL 数据库内核月报 ：http://mysql.taobao.org/monthly/

17. Mini-Project Labs

• DeepDB : https://github.com/deepbodra97/Database-System-Implementation

• Mini-OB : https://open.oceanbase.com/activities/4921877

• CS 15-445 Labs

18. AI4DB and DB4AI (frontier tech)

• University of Magdeburg slides:
  • slide 01 : https://www.dbse.ovgu.de/en/-p-578-EGOTEC-35lsmf8qh52t9v3rhrjdojrd46/_/5_ai-1.pdf
  • slide 02 : https://www.dbse.ovgu.de/-p-578/_/5_ai-2.pdf
• Tsinghua University databaseGroup Github : https://github.com/TsinghuaDatabaseGroup/AIDB
• Tsinghua & MIT Paper : AI Meets Database: AI4DB and DB4AI , SIGMOD2021
• openGauss Blog : openGauss AI4DB and DB4AI
• MIT databaseGroup : http://dsg.csail.mit.edu/mlforsystems/papers/
• Blogger Github : https://github.com/LumingSun/ML4DB-paper-list

Database Retrospective

• Databases in 2021: A Year in Review , Andy Pavlo
• Databases in 2022: A Year in Review , Andy Pavlo