SQLite as an executable format: SELF project swaps ELF for a database

A file that file identifies as SQLite 3.x database, yet still runs when you do ./hello. That's what engineer Farid Zakaria demonstrates in "Your executable is a SQLite database", published in August and discussed on Hacker News. The project is called SELF (Structured Executable & Linkable Format) and proposes replacing ELF, Linux's standard binary format, with a SQLite database.

The idea isn't new for the author: it comes from his doctoral thesis and an earlier tool, sqlelf, which already allowed querying ELF files via SQL (SELECT name FROM elf_symbols instead of combining readelf with grep). The difference now is radical: it isn't a database that describes the executable, it's the file itself that you make executable and run.

$ file hello
hello: SQLite 3.x database, application id 0x53454c46, user version 1

$ ./hello
Hello, world!

$ sqlite3 hello 'SELECT soname FROM ldd'
libc.so.6

The thesis: ELF is already a poorly disguised database

Zakaria's central argument is that ELF reinvents, by hand, primitives that any database already offers. The table he builds in the post is the most provocative part of the text:

• .strtab / .dynstr do string interning
• .hash / .gnu.hash are an index (a CREATE INDEX)
• the section header table is sqlite_schema, a table of tables
• st_name → offset is a hand-rolled foreign key
• objcopy --strip-debug is a DELETE followed by VACUUM

ELF was designed for a world where disk and bandwidth were extremely expensive: it's an extremely compact format, hard to modify (you often need to zero out sections and recreate others because everything is packed tightly) and lacking a self-describing schema. SQLite is the counterexample: a stable, self-describing format, designed to be extended without breaking existing consumers.

What disappears when you swap the format

A SELF file needs only two tables to run: self_meta (the ELF header as key/value pairs) and segments (the load image, one row per program header, with the bytes in a BLOB). The symbols table replaces several ELF sections at once:

CREATE TABLE symbols (
  id INTEGER PRIMARY KEY,
  name TEXT NOT NULL,
  version TEXT,
  value INTEGER, size INTEGER,
  type TEXT, bind TEXT,
  defined INTEGER NOT NULL,
  exported INTEGER NOT NULL
);
CREATE INDEX idx_symbols_name ON symbols (name, version);

With this, .dynstr disappears (name is TEXT and SQLite already interns strings), symbol versioning becomes a column instead of the .gnu.version_r/.gnu.version_d arrangement, and the index is a b-tree maintained by SQLite instead of the hand-rolled bloom filter in .gnu.hash.

The practical consequence is that every tool that reads ELF becomes a query, and every tool that modifies ELF becomes a transaction. Where there used to be fragile offset surgery, now there's SQL:

# strip(1) becomes DELETE + VACUUM
$ sqlite3 hello 'DELETE FROM sections; DELETE FROM notes; VACUUM;'
# 57344 -> 49152 bytes, and it still runs
$ ./hello
Hello, world!

In the author's words, strip is a DELETE, patchelf is an UPDATE, and ldd is a view (a join of the symbols table with the segments). Information missing from the schema can be exposed as a VIEW.

How the kernel runs a SQLite file

The trick lies in application_id, a 4-byte field that SQLite reserves at offset 68 of the header exactly for this kind of use. SELF stamps the SELF signature there, so a regular SQLite database is never mistaken for an executable.

The rest is binfmt_misc, the Linux subsystem that lets you invoke any file as if it were a native binary, by registering a magic number and an interpreter. On NixOS, the configuration is just a few lines matching the SQLite magic at offset 0 and SELF at offset 68. The interpreter is self-exec, a small C program linked against libsqlite3 whose behavior resembles ld.so: it fetches program headers and symbols from the database, maps the segments into memory, performs relocations, and jumps to the entry point. (self-exec needs to remain an ELF itself, otherwise the kernel enters recursion until it hits -ELOOP.)

For dynamic linking, Zakaria tested two paths: using glibc's rtld-audit interface to answer "which library satisfies this symbol?" with a SQL query, keeping lazy PLT, IFUNCs, TLS, and symbol versioning working; and a dynamic linker of his own written entirely in SQL, self-ld, still as a proof of concept.

The price: double the size and lost mmap

This is where the numbers come in that keep it from being production-ready today. A SELF file carries the overhead of SQLite's b-tree and ends up roughly twice as large as the equivalent ELF. Much of that is recoverable with strip: a stripped SELF coreutils came out at 1,794,048 bytes against 1,768,632 for the ELF, a difference below 1%.

The more serious problem is latency. There's a fixed cost of roughly 5 ms to open SQLite and start the interpreter, plus a copy proportional to the image size. And that copy is worse than it sounds: since the bytes come from b-tree pages instead of being mapped, two processes running the same SELF binary don't share text pages the way they would with a normal ELF loaded via mmap. One of the major advantages of the traditional loading model is lost.

The surprising part: an entire userland in one file

The most interesting part of the experiment is the closure concept. Since ldd only lists sonames (it's ambiguous about which specific file satisfies each dependency), SELF stores the resolved path of each edge in the database, turning library resolution into a foreign key and ldd into a JOIN. The self closure command packages a binary and all its transitive dependencies into a single .db. ls plus its five libraries became a 4.8 MiB file.

Taking it to the extreme, the author pointed self closure at every binary in the system's PATH: 723 executables, 400 distinct libraries, 1,123 objects, 346,386 symbols, and 3,808 dependency edges, all in a single SQLite file. And thanks to deduplication, the result ended up even smaller than the sum of the ELFs: 611.9 MiB for the database versus 644.4 MiB for the ELFs.

What this changes (and what remains open)

It's important to separate hype from reality: SELF is a research prototype, available on GitHub, running primarily on NixOS through a postFixup hook that converts ELF to SELF per package. There's no gcc or ld emitting SELF directly yet, the size overhead is real, and the loss of page sharing via mmap is a concrete performance obstacle for the model to generalize.

For those building software in Brazil, the immediate value isn't in swapping the toolchain, but in the provocation: much of the pain of working with binaries (parsers rewritten everywhere, offset surgery, out-of-band index caches like ldconfig and debuginfod) exists because the format isn't queryable. Rethinking the executable as structured, queryable data is an idea that speaks directly to anyone working with reproducible distribution, dependency analysis, and supply chain. It remains open whether the runtime cost can be amortized, whether a 100% SQL linker scales, and whether the industry has the appetite to challenge the inertia of a format that has dominated Linux for decades.