TinyKV - Distributed Key-Value Storage System

A learning implementation of a distributed key-value storage system based on the TinyKV Course

About

This project is an implementation of the TinyKV course, which builds a key-value storage system with the Raft consensus algorithm. The course is inspired by MIT 6.824 and the TiKV Project.

TinyKV helps you understand how to implement a horizontally scalable, highly available, key-value storage service with distributed transaction support, along with a deeper understanding of TiKV architecture and implementation.

Project Status

• Project 1: Standalone KV - ✅ Completed

  • Standalone storage engine implementation
  • Raw key-value service handlers

• Project 2: Raft KV - 🚧 In Progress

  • Basic Raft algorithm
  • Fault-tolerant KV server on top of Raft
  • Raft log garbage collection and snapshot support

• Project 3: Multi-raft KV - 📋 Planned

  • Membership change and leadership change
  • Conf change and region split on Raft store
  • Basic scheduler implementation

• Project 4: Transaction - 📋 Planned

  • Multi-version concurrency control (MVCC) layer
  • Transaction handlers implementation

Course Architecture & Projects

The project consists of four main stages, each building upon the previous one:

Project 1: Standalone KV

Documentation | ✅ Completed

• Implement a standalone storage engine
• Implement raw key-value service handlers

Project 2: Raft KV

Documentation | 🚧 In Progress

• Implement the basic Raft algorithm
• Build a fault-tolerant KV server on top of Raft
• Add support for Raft log garbage collection and snapshot

Project 3: Multi-raft KV

Documentation | 📋 Planned

• Implement membership change and leadership change in Raft
• Implement conf change and region split on Raft store
• Implement a basic scheduler

Project 4: Transaction

Documentation | 📋 Planned

• Implement the multi-version concurrency control (MVCC) layer
• Implement handlers for KvGet, KvPrewrite, and KvCommit requests
• Implement handlers for KvScan, KvCheckTxnStatus, KvBatchRollback, and KvResolveLock requests

Code Structure

Similar to the architecture of TiDB + TiKV + PD that separates storage and computation, TinyKV focuses on the storage layer of a distributed database system. If you're interested in the SQL layer, check out TinySQL. TinyScheduler acts as the central control of the TinyKV cluster, collecting information from heartbeats and generating scheduling tasks. All instances communicate via gRPC.

Directory Structure

├── kv/                  # Key-value store implementation
│   ├── server/          # gRPC server and service handlers
│   ├── storage/         # Storage interface and implementations
│   ├── raftstore/       # Raft-based storage engine
│   ├── transaction/     # Transaction layer (MVCC)
│   └── coprocessor/     # Coprocessor for data processing
├── raft/                # Raft consensus algorithm implementation
├── scheduler/           # TinyScheduler for cluster management
│   ├── server/          # Scheduler server and coordinator
│   └── client/          # Scheduler client
├── proto/               # Protocol Buffers definitions
└── log/                 # Logging utilities

Learning Resources

Prerequisites

• Git: Install Git
• Go: Version ≥ 1.13 (Installation Guide)

Build

Build TinyKV from source:

cd go-distributed-system
make

This builds the tinykv-server and tinyscheduler-server binaries in the bin/ directory.

Running the Server

Run the standalone server:

make
./bin/tinykv-server

Testing

Run tests for specific components:

# Test standalone storage
make project1

# Test Raft implementation
make project2

# Test Multi-Raft
make project3

# Test transactions
make project4

Integration with TinySQL

You can run TinyKV with TinySQL to get a complete distributed database experience:

1. Get tinysql-server following TinySQL documentation
2. Place tinyscheduler-server, tinykv-server, and tinysql-server binaries in the same directory
3. Run the following commands:

mkdir -p data
./tinyscheduler-server
./tinykv-server -path=data
./tinysql-server --store=tikv --path="127.0.0.1:2379"

4. Connect with MySQL client:

mysql -u root -h 127.0.0.1 -P 4000

Learning Resources

Reading List

Check out the reading list for resources on distributed storage systems.

Recommended Reading

• TiKV Design: Data storage architecture
  • English
  • Chinese
• PD Design: Scheduling system
  • English
  • Chinese
• Raft Paper: The Raft Consensus Algorithm
• Raft Visualization: Interactive Raft

Acknowledgments

This project is based on the TinyKV Course by PingCAP, part of the Talent Plan educational initiative.

License

See LICENSE file for details.