A distributed key-value store in Rust. Every tenant gets its own slice of an ordered keyspace, and the rest of the design works out from there.
Early. The single-node binary runs. It speaks gRPC, keeps tenants apart at the keyspace level, has configurable limits, shuts down without losing in-flight requests, and stores data either in memory or on disk through fjall. Everything that would make it actually distributed (Raft, range sharding, transactions) is on the roadmap and tracked as milestones.
A short reality check before going further: Current capabilities lists what works. What is not done yet lists what does not. Do not run this in production.
The whole design starts from one decision: tenants belong inside the keyspace, not on top of it. Every key carries a tenant prefix. That means a tenant's data is contiguous on disk and contiguous across shards, range scans inside a tenant stay local to a small number of shards, and per-tenant accounting (bytes used, ops/second) falls out for free. Tenants do not get separate processes or VMs; the isolation is logical and that is enough.
The second piece is the provisioning model. Picking instance sizes and disk volumes for a database is a bad time, so the goal is to not ask. Nodes join. Shards split when they grow past a target size. The placement layer moves things around. The cluster has whatever capacity it has.
If this reminds you of FoundationDB, that is the point. The keyspace model is borrowed. The difference is that FerrisKV does not sit on top of FoundationDB; it tries to reach a similar surface on a pure-Rust stack: fjall for local storage, raft-rs for consensus, tonic for gRPC.
Short version below. Longer write-up in docs/architecture.md.
+---------------------------------+
| Coordinator cluster (3 to 5) |
| Raft-replicated metadata: |
| placement table, schemas, |
| timestamp oracle |
+-----------------+---------------+
|
+-----------------+---------------+
| Storage nodes |
| Per-shard Raft groups |
| Local engine: fjall (LSM) |
| MVCC + Percolator on top |
+-----------------+---------------+
|
+-----------------+---------------+
| Clients (CLI, gRPC, eventually |
| SQL via DataFusion) |
+---------------------------------+
What exists today is the scaffolding. The Raft and transactional code lives in proto definitions and roadmap issues, not in running code.
Keys have structure. Encoding details in docs/keyspace.md. The summary:
[tenant length: u8] [tenant bytes] [subspace: u8] [user key]
Four subspaces per tenant:
| Byte | Subspace | What lives here |
|---|---|---|
| 0 | metadata | Schemas, configuration, ACLs |
| 1 | data | User records, keyed by primary key |
| 2 | index | Secondary indexes for non-primary lookup |
| 3 | stats | Storage usage and telemetry |
Lexicographic order is preserved, so a tenant's data is one contiguous range and a prefix scan over 1_data only reads inside that tenant.
The server handles gRPC get, put, delete, streamed scan, and batch. Tenant isolation is enforced at the keyspace level. Two storage backends are configurable: an in-memory DashMap for development, or fjall for persistence. The write-ahead log on disk recovers its sequence counter at boot. Configurable limits on key size, value size, batch size, and scan cap. Per-request TTL is enforced by an index-driven background sweeper. JWT authentication and per-RPC RBAC are enforced on every call. TLS is supported with configurable cert and key paths. A structured audit log records writes. An admin HTTP server exposes /healthz, /readyz, and Prometheus /metrics. Shutdown handles SIGINT and SIGTERM, drains in-flight requests, and flushes the WAL before exiting. The workspace currently has 105 tests covering the codec, the storage backends, the placement structure, the gRPC handlers, the TTL sweeper, and the auth primitives.
No replication, so losing the single node loses data. No multi-key transactions, no MVCC, no snapshot reads. The Watch RPC returns Unimplemented. No range sharding or rebalancing across nodes. No client libraries outside Rust, no SQL surface, no Kubernetes operator or Helm chart. No automated backups to object storage.
All of it is tracked as issues, grouped under the milestones.
Build the release binaries.
cargo build --release -p ferriskv-node -p ferriskv-cliWrite a minimal config.
mkdir -p config data
cat > config/node.toml <<'EOF'
node_id = "node-0"
listen = "127.0.0.1:7100"
data_dir = "./data/node"
coord_endpoints = []
backend = "fjall"
shutdown_timeout_secs = 30
[limits]
max_key_size = 4096
max_value_size = 10485760
max_batch_ops = 1000
max_scan_limit = 10000
EOFStart the node.
./target/release/ferriskv-node --config config/node.tomlSet admin_listen in node.toml to start a secondary HTTP server with two routes:
admin_listen = "127.0.0.1:7101"GET /healthzreturns200 OKwhile the process is alive.GET /readyzreturns200 OKwhen the storage engine answers a probe read,503otherwise.GET /metricsreturns Prometheus text format with RPC latencies and counters, value size histograms, and audit event counts.
Bind to 127.0.0.1 (the default convention) so the routes stay reachable only from the same host: Kubernetes probes, sidecars and local agents have access, the outside network does not. If you genuinely need cross-host scraping, switch to 0.0.0.0 and pair it with a NetworkPolicy or firewall rule. The server logs a warning at startup when it binds outside loopback.
Leaving admin_listen unset disables the admin server entirely.
Two settings, one per request, one server-wide. They are independent.
Per-request, on the put: ttl_ms. The lifetime of that specific key in milliseconds. 0 (the default) means the key never expires. Set it via the CLI flag --ttl-ms or directly in PutRequest.ttl_ms from a gRPC client. Stored values are framed with a one-byte version prefix; setting a TTL adds an 8-byte expiration timestamp on top.
Server-wide, in node.toml: ttl_sweep_interval_secs. Maximum interval between two passes of the background TTL sweeper, in seconds. Default 60. Setting it to 0 disables the sweeper entirely, in which case expired values stay on disk but remain invisible to reads (a get on an expired key still returns "not found").
ttl_sweep_interval_secs = 60Internally the node keeps an in-memory index of keys with a scheduled expiration. The sweeper wakes exactly when the next key is due, not on a fixed tick, so cleanup happens at the millisecond after expiration in the common case. The interval above is only an upper bound used as a fallback (so the loop still runs from time to time even when no key has a TTL). Expired keys disappear from reads immediately regardless of when the sweeper runs.
Quick test:
# 5-second TTL
./target/release/ferriskv --tenant alice put hello world --ttl-ms 5000
./target/release/ferriskv --tenant alice get hello # world
sleep 6
./target/release/ferriskv --tenant alice get hello # exits 1The GC currently does a full keyspace scan per pass, which is fine for the scale targeted in P0. An indexed scan keyed by expiration time is planned for later.
The server listens in plaintext by default. To enable TLS, add a [tls] section to node.toml pointing at a certificate and key in PEM format:
[tls]
cert_path = "/etc/ferriskv/server.crt"
key_path = "/etc/ferriskv/server.key"For local development, mkcert is the fastest way to get a trusted certificate:
mkcert -install
mkcert -cert-file server.crt -key-file server.key localhost 127.0.0.1For production, point at certificates issued by Let's Encrypt or your internal CA. The node does not handle certificate renewal; reload by restarting the process or use a sidecar that watches the file and signals the node.
Use the CLI.
./target/release/ferriskv --tenant alice put hello world
./target/release/ferriskv --tenant alice get hello
./target/release/ferriskv --tenant alice scan ""
./target/release/ferriskv --tenant alice delete helloA different tenant sees none of alice's keys, even when the user-supplied key is identical:
./target/release/ferriskv --tenant bob get hello| Crate | Role |
|---|---|
ferriskv-core |
Key codec, storage trait, error types, limits, hashing |
ferriskv-proto |
Protobuf generated by tonic-build (pure Rust via protox) |
ferriskv-auth |
JWT verification, RBAC, API key store |
ferriskv-coord |
Coordinator binary and placement code |
ferriskv-node |
Storage node binary, WAL, gRPC service |
ferriskv-cli |
Command-line client |
make test # cargo test --workspace
make clippy # cargo clippy --workspace --all-targets -- -D warnings
make fmt # cargo fmt --all
make build # cargo build --workspace --all-targetsCI runs the same on every push and pull request.
Six phases, each a GitHub milestone.
| Phase | Theme |
|---|---|
| P0 | Single-node production readiness |
| P1 | Distribution (Raft, sharding) |
| P2 | Transactions (MVCC, Percolator) |
| P3 | Multi-tenant database features |
| P4 | SQL and ecosystem |
| P5 | Maturity (tests, fuzzing, docs) |
The intent behind each phase is in docs/roadmap.md.
Apache-2.0.