Skip to content

zmeadows/tskv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

7 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

⏳︎ tskv β€” Time-Series Key-Value Store

TL;DR: Single-node, crash-safe time-series KV store with a non-blocking TCP server (Linux epoll) and LSM-style storage: Write-Ahead Log (WAL) β†’ memtable β†’ immutable SSTables, plus a background compaction worker. Built with C++23 modules; no third-party libraries.


β—Ž Goals

  • Demonstrate disciplined systems design in modern C++.
  • Show clear durability boundaries (WAL append / optional sync) and read-after-write visibility.
  • Keep backpressure and buffers bounded for predictable latency.
  • Favor correctness + measurable performance over feature breadth.

β›Ά Architecture

  • Write path: append to WAL β†’ (optional fdatasync) β†’ apply to memtable β†’ periodic flush to SSTable (immutable, sorted).
  • Read path: memtable first β†’ then newest-to-oldest SSTables; per-file Bloom filter to skip negatives; index to jump to the right block.
  • Compaction: merge overlapping SSTables, keep newest versions, drop obsolete ones; install via manifest with durable rename.

βš‘ Roadmap (high-level)

  • v0.1 β€” Bootstrap: README, CLI, PR template
  • v0.2 β€” Non-blocking TCP + epoll echo; clean shutdown
  • v0.3 β€” Framing: header + length; PING/PONG
  • v0.4 β€” Connection state: RX/TX rings; backpressure cap
  • v0.5 β€” Engine queues: SPSC/MPSC; dispatcher
  • v0.6 β€” WAL v1: append+CRC; sync policy flag
  • v0.7 β€” Recovery: replay WAL; torn-tail safe
  • v0.8 β€” Memtable v0: std::map; PUT/GET end-to-end
  • v0.9 β€” SSTable v1: writer/reader; mmap; footer
  • v0.10 β€” Manifest: live tables; durable rename
  • v0.11 β€” Wire-through: GET/PUT via SST path
  • v0.12 β€” Bloom filters: per-SST; bits/key tuning
  • v0.13 β€” Memtable v1: skiplist + iterator
  • v0.14 β€” SCAN RPC: streaming RESP; writev batches
  • v0.15 β€” Concurrency: N I/O, M engine; fairness
  • v0.16 β€” Metrics: counters + p50/p95/p99 endpoint
  • v0.17 β€” Compaction v1: merge + manifest install
  • v0.18 β€” Chaos tests: disk-full; kill-9 loops
  • v0.19 β€” Perf pass: micro/macro benches; notes
  • v1.0 β€” Polish: docs, demo.sh, ASan/UBSan; release

∷ C++ Module Layout

  • tskv.common.* β€” logging, metrics, ring buffers, fs helpers
  • tskv.net.* β€” socket (non-blocking), reactor (epoll, edge-triggered), connection, rpc
  • tskv.kv.* β€” engine, wal, memtable, sstable, manifest, compaction, filters

βˆ‘ Metrics (planned)

  • net: connections_open, rx_bytes_total, tx_bytes_total, backpressure_events_total
  • rpc: put_total, get_total, scan_total, errors_total
  • wal/sstable: appends_total, fsync_total, files_total, bloom_negative_total
  • latency: p50/p95/p99 for GET & PUT

About

Single-node, in-memory time-window cache for time-series data

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors