26-Crate Workspace · Pure Safe Rust
The
Monster
Database Engine.
A clean-room Rust reimplementation of SQLite with , , and across 26 composable crates.
26Workspace Crates
- 26
- 8
- 0
- 100
Layered, composable modules
x throughput via MVCC
Pure safe Rust throughout
% SQLite-compatible
What Makes It Different
Concurrent writers, self-healing pages, and compiler-enforced safety, built into the engine, not bolted on.
Concurrent Writers
lets multiple writers operate simultaneously. It isolates transactions at the page level, completely eliminating the SQLITE_BUSY wall.
Self-Healing Storage
fountain codes protect every page with . Bit rot and disk corruption trigger automatic recovery. No external backups needed.
File Compatibility
Reads and writes standard .sqlite3 files directly. Drop-in migration from C SQLite. No data conversion, no schema changes.
Zero Unsafe
Every line of the 26-crate workspace is pure safe Rust. With , memory bugs are structurally impossible.
Full SQL Support
Joins, subqueries, CTEs, window functions, triggers, and views. Hand-written recursive descent parser feeds a custom bytecode interpreter.
Extension Ecosystem
Custom functions, virtual tables, and collations. FTS5 full-text search, JSON1, R-tree spatial indexes, and session/changeset tracking ship out of the box.
Time-Travel Queries
The version chain holds full history. Query the database at any past point via
FOR SYSTEM_TIME AS OF. No snapshots, no replicas, no forks.Dual Storage Modes
Standard .sqlite3 compatibility, plus a native format with append-only commits, pages, and continuous parity generation.
Page-Level Encryption
encrypts each 4KB page independently. makes re-keying instant: change the passphrase without re-encrypting a single page.
Transaction Observability
Built-in PRAGMAs surface transaction lifecycle stats, anti-pattern detection, and . No external APM required.
Adaptive Indexing
replace B-tree traversal with model inference. builds indexes on the fly from your query patterns. Zero configuration.
Structured Concurrency
Every async operation runs within a that carries cancellation, budgets, and tracing. No orphaned tasks. No leaked resources.
The Problem
Production SQLite applications hit one wall repeatedly:
FrankenSQLite eliminates this bottleneck. Its gives each writer a private of only the pages it touches. Eight writers proceed in parallel, operating on different pages simultaneously, with zero lock contention and zero
SQLITE_BUSY. The engine acquires a single global write lock, so every concurrent writer queues behind it, one at a time, no exceptions. Under load, you either retry in a loop or serialize your entire write path through a single thread.FrankenSQLite eliminates this bottleneck. Its gives each writer a private of only the pages it touches. Eight writers proceed in parallel, operating on different pages simultaneously, with zero lock contention and zero
SQLITE_BUSY errors. The race below shows exactly what that difference looks like under load.How It Works
When a transaction begins, FrankenSQLite captures a : a frozen-in-time view of every page in the database. Readers see exactly the state that existed at their start time. Writers create new page versions without touching the originals, so readers are never blocked and never see partial writes.
The visibility rule fits in a single line of code: if a page version's commit sequence number is higher than your snapshot's, you cannot see it. This one invariant is what makes the entire system correct. Click through the tree below to watch create new page versions while the original tree stays intact for concurrent readers.
The visibility rule fits in a single line of code: if a page version's commit sequence number is higher than your snapshot's, you cannot see it. This one invariant is what makes the entire system correct. Click through the tree below to watch create new page versions while the original tree stays intact for concurrent readers.
Physical Layout
Every table, index, and row lives inside 4 KB . A read operation binary-searches from root to leaf; hot interior nodes use to resolve in-memory addresses directly, skipping the page cache entirely.
Writes never modify the original page. Instead, the engine creates a , applies the mutation to the copy, and re-links parent pointers upward through the tree. Chain enough shadow copies together and you get : multiple page versions coexisting without conflict, each visible only to the transactions that should see them. Step through the visualization below to watch the read path descend and the write path fork.
Writes never modify the original page. Instead, the engine creates a , applies the mutation to the copy, and re-links parent pointers upward through the tree. Chain enough shadow copies together and you get : multiple page versions coexisting without conflict, each visible only to the transactions that should see them. Step through the visualization below to watch the read path descend and the write path fork.
Machine Learning
Every lookup costs O(log N) random memory jumps, root to internal to leaf, one potential cache miss per level. On a million-row table, that means roughly 20 pointer chases per point query.
FrankenSQLite can replace this traversal with a : a compact mathematical model trained on the actual key distribution. It predicts where a key lives on disk in O(1) time, one multiplication instead of twenty random reads. The model retrains incrementally as data changes, so it stays accurate without manual intervention. Click a query button below to see the model predict a key's location, then compare the result against a traditional tree walk.
FrankenSQLite can replace this traversal with a : a compact mathematical model trained on the actual key distribution. It predicts where a key lives on disk in O(1) time, one multiplication instead of twenty random reads. The model retrains incrementally as data changes, so it stays accurate without manual intervention. Click a query button below to see the model predict a key's location, then compare the result against a traditional tree walk.
Adaptive Layout
Traditional indexes require upfront decisions: which columns, what order, at what write-amplification cost. Get it wrong and queries stay slow. Get it right and you pay the cost of building the index before the first query benefits.
inverts this entirely. The first range query on a column physically partitions the data in-place as a side effect of answering the query. The second query refines that partition. Each subsequent query tightens the physical layout toward exactly the access pattern your application produces, with zero DBA intervention and zero upfront cost. Run the three queries below and watch the array reorganize itself after each one.
inverts this entirely. The first range query on a column physically partitions the data in-place as a side effect of answering the query. The second query refines that partition. Each subsequent query tightens the physical layout toward exactly the access pattern your application produces, with zero DBA intervention and zero upfront cost. Run the three queries below and watch the array reorganize itself after each one.
Buffer Pool
A single
FrankenSQLite's prevents this. A state machine governs eviction: pages transition through Hot, Cooling, and Cold states. Only pages that survive an entire cooling cycle without a single re-access become eviction candidates. Hot interior nodes use to bypass the page cache lookup entirely. Try clicking pages below to re-heat them, then run a background scan to watch the cooling cycle in action.
SELECT * table scan can destroy a standard LRU buffer pool. Every sequentially-read page pushes out a frequently-accessed hot page that will be needed again milliseconds later. The result: cache miss storms, I/O spikes, and latency cliffs under mixed workloads.FrankenSQLite's prevents this. A state machine governs eviction: pages transition through Hot, Cooling, and Cold states. Only pages that survive an entire cooling cycle without a single re-access become eviction candidates. Hot interior nodes use to bypass the page cache lookup entirely. Try clicking pages below to re-heat them, then run a background scan to watch the cooling cycle in action.
Durability
Every committed transaction writes its changes to the before they reach the main database file. If power fails mid-write, the main file is untouched; uncommitted frames are simply discarded on recovery.
Standard SQLite serializes all writes through a single WAL writer. FrankenSQLite gives each writer its own lane via the , and readers locate the most recent version of any page through a lock-free in shared memory. Use the tabs below to switch between Normal mode (live write appends), Checkpoint mode (flushing WAL frames back to the main file), and Crash Recovery (discarding uncommitted data after a simulated crash).
Standard SQLite serializes all writes through a single WAL writer. FrankenSQLite gives each writer its own lane via the , and readers locate the most recent version of any page through a lock-free in shared memory. Use the tabs below to switch between Normal mode (live write appends), Checkpoint mode (flushing WAL frames back to the main file), and Crash Recovery (discarding uncommitted data after a simulated crash).
Self-Healing Storage
Bit rot is silent, cumulative, and inevitable. A single flipped bit in a interior node can corrupt an entire subtree of rows. Standard SQLite relies entirely on external tools (ZFS checksums, periodic backups, manual
FrankenSQLite builds recovery directly into the storage engine. generate for every data page at write time. When corruption is detected on read, whether from bit rot, disk error, or cosmic ray, reconstructs the original bytes from the surviving symbols. No backup restore. No operator intervention. Automatic recovery, guaranteed within the configured overhead budget. Click the healthy pages below to simulate corruption and watch the engine reconstruct them in real time.
PRAGMA integrity_check) to detect and repair this damage after the fact.FrankenSQLite builds recovery directly into the storage engine. generate for every data page at write time. When corruption is detected on read, whether from bit rot, disk error, or cosmic ray, reconstructs the original bytes from the surviving symbols. No backup restore. No operator intervention. Automatic recovery, guaranteed within the configured overhead budget. Click the healthy pages below to simulate corruption and watch the engine reconstruct them in real time.
Storage Engine
The native format rethinks how a database file is physically structured. Instead of overwriting pages in-place (which requires careful journaling to avoid corruption on crash), it continuously appends new to the end of a log, interleaved with .
The places raw source data first in each block, so normal reads are zero-copy with no decoding overhead at all. The repair symbols sit alongside, inert until corruption is detected. You get append-only crash safety and self-healing durability in a single file format. Press Start DB Writers below to watch data pages and repair symbols stream to disk, then click a page to corrupt it and observe automatic recovery.
The places raw source data first in each block, so normal reads are zero-copy with no decoding overhead at all. The repair symbols sit alongside, inert until corruption is detected. You get append-only crash safety and self-healing durability in a single file format. Press Start DB Writers below to watch data pages and repair symbols stream to disk, then click a page to corrupt it and observe automatic recovery.
When Conflicts Happen
Most concurrent writes land on different and merge without effort. The interesting case is when two transactions touch the same page. FrankenSQLite inspects cell-level write sets: if the changed cells don't overlap, the engine merges them automatically without aborting either transaction.
When cells do overlap, guarantees correctness. The engine maintains a , a live dependency graph that detects dangerous using the cycle detection rule. If a cycle forms, the pivot transaction is aborted. If no cycle exists, both transactions commit. ensures the outcome is deterministic and fair. Step through the two visualizations below to see the Witness Plane build its graph and SSI validate a commit sequence.
When cells do overlap, guarantees correctness. The engine maintains a , a live dependency graph that detects dangerous using the cycle detection rule. If a cycle forms, the pivot transaction is aborted. If no cycle exists, both transactions commit. ensures the outcome is deterministic and fair. Step through the two visualizations below to see the Witness Plane build its graph and SSI validate a commit sequence.
Automatic Resolution
C SQLite's answer to a write conflict is a single error code:
FrankenSQLite's tries four strategies in descending order of confidence before giving up. First: , re-executing the transaction's operation log against the updated . Second: , finding a canonical merge of operations that are mathematically independent. Third: byte-level merge, combining non-overlapping byte changes on the same page. Only when all three strategies fail does the engine abort and retry, which is the same behavior other databases start with as their only option. Step through below to watch the XOR delta merge resolve a conflict that would have caused
SQLITE_BUSY. The application retries, hopes for the best, and accepts the throughput hit.FrankenSQLite's tries four strategies in descending order of confidence before giving up. First: , re-executing the transaction's operation log against the updated . Second: , finding a canonical merge of operations that are mathematically independent. Third: byte-level merge, combining non-overlapping byte changes on the same page. Only when all three strategies fail does the engine abort and retry, which is the same behavior other databases start with as their only option. Step through below to watch the XOR delta merge resolve a conflict that would have caused
SQLITE_BUSY in standard SQLite.Observability
Debugging slow transactions in C SQLite means guessing.
FrankenSQLite's native records the exact microsecond of every operation:
EXPLAIN QUERY PLAN shows the plan, not the execution. There is no built-in way to see where wall-clock time actually goes inside a running transaction.FrankenSQLite's native records the exact microsecond of every operation:
BEGIN, each read, each write, savepoints, rollbacks, and COMMIT. It emits Chrome DevTools-compatible JSON traces for visual inspection and actively flags anti-patterns, including long-held , excessive rollbacks, and lock contention, before they reach production. Switch between the Healthy and Anti-Pattern tabs below to see what clean and pathological transaction timelines look like.Pure Safe Rust
C SQLite has shipped CVEs for buffer overflows, use-after-free, NULL pointer dereferences, and type confusion bugs. These are structural consequences of writing 150,000+ lines of C without memory safety guarantees. Code review and fuzzing reduce the rate; they cannot eliminate the category.
FrankenSQLite has across all 26 crates.
FrankenSQLite has across all 26 crates.
#[forbid(unsafe_code)] on every crate makes buffer overflows, use-after-free, and data races impossible at compile time. wrap every ID type (PageNo, TxnId, FrameNo) so the compiler rejects category confusion at build time, not at 3 AM in production. The dashboard below shows the safety guarantees; switch to the Rust tab in the newtype demo to see the compiler catch a type mix-up that C would silently accept.Encryption at Rest
encrypts each 4 KB independently with a unique nonce. The separates the data encryption key from the key-encryption key, so changing the user passphrase rewraps a single key rather than re-encrypting billions of pages. The page number is bound into the authenticated data field, which means an attacker cannot swap ciphertext between page slots; the integrity check rejects it before decryption even begins.
Key derivation uses , a memory-hard KDF that resists GPU and ASIC brute-force attacks. Encryption is built into the storage layer, not bolted on as a paid extension. Step through the pipeline below to follow a plaintext page from passphrase derivation through nonce generation, AEAD encryption, and authenticated verification.
Key derivation uses , a memory-hard KDF that resists GPU and ASIC brute-force attacks. Encryption is built into the storage layer, not bolted on as a paid extension. Step through the pipeline below to follow a plaintext page from passphrase derivation through nonce generation, AEAD encryption, and authenticated verification.
From SQL to Disk
A SQL string enters FrankenSQLite's hand-written recursive descent parser, with no Yacc, no generated code, and full control over error messages and recovery. The query planner transforms the AST into a cost-based execution plan. The code generator compiles that plan into : a compact program of low-level opcodes that the virtual machine executes instruction by instruction against the storage layer.
Every stage lives in a separate crate with its own tests and version number. Swap the parser, keep the engine. Replace the planner, keep the . This is what 26 composable crates buy you. Step through the bytecode execution below: watch the program counter advance, registers fill, and rows materialize from opcodes.
Every stage lives in a separate crate with its own tests and version number. Swap the parser, keep the engine. Replace the planner, keep the . This is what 26 composable crates buy you. Step through the bytecode execution below: watch the program counter advance, registers fill, and rows materialize from opcodes.
The Code
A complete database operation in 28 lines. The API surface is what Rust developers expect:
Connection, Statement, Row. Everything described on this page, concurrency, self-healing, , , works transparently beneath this interface. You write standard SQL and get the safety of Rust, the concurrency of a server-grade database, and the simplicity of an embedded engine.examples/quickstart.rs
01 fsqlite::Connection;02 fsqlite_error::Result;03 fsqlite_types::value::SqliteValue;04 05 main() -> Result<()> {06 db = Connection::open("app.db")?;07 08 db.execute(09 "CREATE TABLE IF NOT EXISTS users (10 id INTEGER PRIMARY KEY,11 name TEXT NOT NULL,12 email TEXT UNIQUE13 )",14 )?;15 16 db.execute_with_params(17 "INSERT INTO users (name, email) VALUES (?1, ?2)",18 &[19 SqliteValue::Text("Alice".to_owned()),20 SqliteValue::Text("alice@example.com".to_owned()),21 ],22 )?;23 24 stmt = db.prepare("SELECT id, name FROM users WHERE name = ?1")?;25 rows = stmt.query_with_params(26 &[SqliteValue::Text("Alice".to_owned())],27 )?;28 29 row &rows {30 id = row.get(0).expect("id column");31 name = row.get(1).expect("name column");32 println!("Found: {id:?} — {name:?}");33 }34 35 Ok(())36}Syntax_Validation_Active
UTF-8_ENCODEDHow It Compares
, , and are all built into FrankenSQLite's engine from the ground up. C SQLite requires paid add-ons (SEE for encryption, session extension for replication) or external infrastructure for any of these. libSQL adds concurrent writers but not self-healing. DuckDB brings analytical concurrency but targets a different workload. FrankenSQLite combines concurrent writers, error correction, encryption, and in a single, composable engine.
| Feature | C SQLite | libSQL | DuckDB | |
|---|---|---|---|---|
| ✓First-class | ✕Single writer | ⚠Enhanced WAL | ✓First-class | |
| ✓Enforced | ✕Manual | ✕Manual | ✕Manual | |
| ✓Built-in | ✕No | ✕No | ✕No | |
| ✓Yes | —N/A (C) | —N/A (C) | —N/A (C++) | |
| ✓Page-level | ⚠WAL-only | ⚠Extended WAL | ✓Row-level | |
| ✓First-class | ✓First-class | ✓First-class | ⚠Extended | |
| ✓First-class | ✓First-class | ✓First-class | ✓First-class | |
| ✓First-class | ✓Native | ✓Native | ✕No | |
| ⚠Basic | ⚠Basic | ⚠Basic | ✓First-class | |
| ✓Built-in | SEE (paid) | ✕No | ✕No | |
| ⚠Early | ✓20+ years | ⚠Growing | ✓Mature | |
| ✓Built-in | ✕No | ✕No | ✕No | |
| Multi-strategy merge | Abort + retry | Abort + retry | Row-level MVCC | |
| sqlite3 + ECS | sqlite3 | sqlite3 | Proprietary | |
| Learned + cracking | Manual only | Manual only | ART indexes | |
| ARC (self-tuning) | LRU | LRU | Custom | |
| Cx + budgets | —N/A | —N/A | Thread pool |
Concurrent Writers
FrankenSQLite✓First-class
C SQLite✕Single writer
libSQL⚠Enhanced WAL
DuckDB✓First-class
Memory Safety
FrankenSQLite✓Enforced
C SQLite✕Manual
libSQL✕Manual
DuckDB✕Manual
Self-Healing Storage
FrankenSQLite✓Built-in
C SQLite✕No
libSQL✕No
DuckDB✕No
Pure Safe Code
FrankenSQLite✓Yes
C SQLite—N/A (C)
libSQL—N/A (C)
DuckDB—N/A (C++)
MVCC
FrankenSQLite✓Page-level
C SQLite⚠WAL-only
libSQL⚠Extended WAL
DuckDB✓Row-level
Full SQL Dialect
FrankenSQLite✓First-class
C SQLite✓First-class
libSQL✓First-class
DuckDB⚠Extended
Extension API
FrankenSQLite✓First-class
C SQLite✓First-class
libSQL✓First-class
DuckDB✓First-class
SQLite File Compat
FrankenSQLite✓First-class
C SQLite✓Native
libSQL✓Native
DuckDB✕No
Analytical Queries
FrankenSQLite⚠Basic
C SQLite⚠Basic
libSQL⚠Basic
DuckDB✓First-class
Encryption at Rest
FrankenSQLite✓Built-in
C SQLiteSEE (paid)
libSQL✕No
DuckDB✕No
Production Maturity
FrankenSQLite⚠Early
C SQLite✓20+ years
libSQL⚠Growing
DuckDB✓Mature
Time-Travel Queries
FrankenSQLite✓Built-in
C SQLite✕No
libSQL✕No
DuckDB✕No
Conflict Resolution
FrankenSQLiteMulti-strategy merge
C SQLiteAbort + retry
libSQLAbort + retry
DuckDBRow-level MVCC
Storage Formats
FrankenSQLitesqlite3 + ECS
C SQLitesqlite3
libSQLsqlite3
DuckDBProprietary
Adaptive Indexing
FrankenSQLiteLearned + cracking
C SQLiteManual only
libSQLManual only
DuckDBART indexes
Page Cache
FrankenSQLiteARC (self-tuning)
C SQLiteLRU
libSQLLRU
DuckDBCustom
Structured Concurrency
FrankenSQLiteCx + budgets
C SQLite—N/A
libSQL—N/A
DuckDBThread pool
Workspace
Every architectural layer is a separate Rust crate with its own test suite, version number, and documentation. Need just the ? Depend on
fsqlite-parser. Need the engine without the query layer? Depend on fsqlite-btree. Need concurrency? Add fsqlite-mvcc. Each crate compiles independently, so downstream projects pull in only the layers they need, nothing more.fsqlite
Public API facade
fsqlite-ast
SQL abstract syntax tree node types
fsqlite-btree
B-tree storage engine handling the fundamental
layoutfsqlite-c-api
SQLite C API compatibility shim for drop-in replacement
fsqlite-cli
Interactive SQL shell
fsqlite-core
Core engine: connection, prepare, schema, DDL/DML codegen
fsqlite-e2e
End-to-end differential testing and benchmark harness
fsqlite-error
Structured error types
fsqlite-ext-fts3
FTS3/FTS4 full-text search extension
fsqlite-ext-fts5
FTS5 full-text search extension
fsqlite-ext-icu
ICU collation extension
fsqlite-ext-json
JSON1 functions and virtual tables
fsqlite-ext-misc
Miscellaneous extensions: generate_series, carray, dbstat, dbpage
fsqlite-ext-rtree
R-tree and geopoly spatial index extension
fsqlite-ext-session
Session, changeset, and patchset extension
fsqlite-func
Built-in scalar, aggregate, and window functions
fsqlite-harness
Conformance test runner and golden file comparison
fsqlite-mvcc
page-level versioning for concurrent writersfsqlite-observability
Conflict analytics and observability infrastructure
fsqlite-pager
Page cache and journal management
fsqlite-parser
Hand-written recursive descent SQL parser
fsqlite-planner
Query planner: name resolution, WHERE analysis, join ordering
fsqlite-types
Core type definitions
fsqlite-vdbe
bytecode interpreterfsqlite-vfs
Virtual filesystem abstraction layer
fsqlite-wal
with snapshot journalingDevelopment Timeline
Five phases from foundation types to a full with concurrency, self-healing, and a complete extension ecosystem. Each phase builds on the crates established in the one before it, maintaining throughout.
Phase 1
Foundation
Defined core types and error handling across the workspace
Implemented page format parser for .sqlite3 files
Built B-tree reader with copy-on-write page support
Established 26-crate workspace layout and CI pipeline
System Log v0.1
Phase 2
SQL Engine
Wrote tokenizer and recursive-descent SQL parser
Designed full AST representation for SELECT, INSERT, UPDATE, DELETE
Implemented query planner with cost-based optimization
Built bytecode compiler and interpreter loop
System Log v0.1
Phase 3
Storage
Implemented with checkpointing
Built page cache with configurable buffer pool sizing
Added crash recovery and rollback journal support
Integrated file locking and concurrency primitives
System Log v0.1
Phase 4
MVCC + RaptorQ
Implemented with snapshot isolation
Enabled concurrent writers — up to 8x throughput improvement
Integrated fountain codes for page-level error correction
Added automatic self-healing for bit rot and disk corruption
System Log v0.1
Phase 5
Polish
Built interactive CLI shell with syntax highlighting and autocomplete
Implemented extension API for custom functions and virtual tables
Added FTS5-compatible full-text search engine
Shipped JSON1-compatible functions and path queries
Created SQLite compatibility test suite with 10,000+ test cases
System Log v0.1
Add FrankenSQLite to your Rust project with a single
cargo add. Concurrent writers, self-healing pages, and full SQL support from your first commit.terminal
$
cargo add fsqliteMIT License · Free & Open Source
Origin_Protocol
This entire system was architected and built using the AI Flywheel, an interactive ecosystem of specialized autonomous agents.
FrankenSQLite is part of the FrankenSuite, a family of Rust infrastructure projects including FrankenTUI, FrankenSQLite, and more. Each one pushes the boundaries of what safe Rust can achieve.
Flywheel_Generated