How to migrate legacy code to Rust incrementally, function by function

Rewriting an entire system in Rust is a recurring temptation for anyone who sees the language deliver performance far above Python, Ruby, or Node. In the talk Accelerating Performance by Incrementally Integrating Rust Into Existing Codebase, at QCon San Francisco, Lily Mara, staff engineer on Discord's Notifications Platform team and author of the book Refactoring to Rust, argues that this impulse usually backfires, and proposes a much more granular path than breaking the monolith into microservices.

The proposal is what she calls FFI refactoring: instead of rewriting the service or even the whole system, you swap the implementation of one specific function for Rust and connect the two languages through the C function interface (the C Foreign Function Interface). The technical argument is simple: practically every operating system and every mainstream language knows how to call a C function, because there is a mountain of C code running in the world. That's the lingua franca that lets you plug Python into Rust in a performant way.

Why full rewrites tend to fail

Mara is direct about the problem with full rewrites, drawing on her own experience with Rust since 2019 (and more than a decade of use if you count side projects). According to her, complete rewrites:

• Frequently blow past deadlines, because they turn out to be more complex than expected;
• Introduce new bugs, or reintroduce old bugs that the legacy system had already fixed;
• Ignore the historical and institutional knowledge embedded in the old code. In her words, old code isn't bad just because it's old: it tends to be complicated because it deals with a complicated set of real-world constraints.

Another point: those who rewrite aiming only at performance tend to look only at the language. And the cost per CPU line is only part of the bill. Changes to architecture, database schema, query patterns, caching layers, and the various layers of microservices tend to weigh as much or more than the language itself.

What makes a good candidate

The strategy isn't worth applying to everything, and Mara's criterion is aggregate: look at where the application spends most of its time. That can be an expensive operation that happens once in a while, or a cheap operation that happens all the time.

The example she gives is telling for anyone who maintains a platform: the request-checking code that sits in front of every API handler. It's not heavy per call, but it runs on every request. If 10% of servers' execution time is in that internal logic, dropping it to 1% (or half a percent) is not a trivial cost reduction in a large organization. That's exactly the kind of infrastructure gain the talk promises, without the overhead of creating a microservice.

To decide where to apply it, Mara uses two axes: Rust's performance against the current language, and tooling support for FFI refactoring on the Rust side.

| Language | Performance gain with Rust | Rust FFI tooling | |---|---|---| | Python, Ruby, Node.js, Lua | High | Excellent (best candidates) | | Go | Rust is usually faster | Weak (runtime limits C FFI) | | C, C++ | Similar (C is sometimes slightly faster) | Good, attractive for memory safety |

The detail about Go deserves attention from anyone with services in that language: C FFI support is limited by the runtime, which makes many assumptions about how to yield execution and needs to suspend part of that when calling a C function, with a per-call cost due to stack size inflation. For C and C++, on the other hand, the draw isn't speed but Rust's memory safety guarantees, which is why projects like Android and the Windows and Linux kernels have been adopting the language.

The tradeoffs you'll pay for

There's no silver bullet, and Mara is honest about the price:

• More complex deployment. Out goes the model of pushing dynamic code and reloading a systemd service; in comes custom native code in the pipeline.
• More complex dev environment. Either you put the Rust compiler on developers' machines, or you ship native binaries that need to match each machine's OS and microarchitecture, or you maintain two implementations in parallel (the old dynamic one and the new Rust one).
• Bug risk. It exists, though smaller than in a giant rewrite, precisely because it operates at a smaller, more restricted scale.

What it looks like in practice: Python + PyO3

The concrete example starts from a Flask application with an endpoint that performs statistical calculations on a list of numbers coming from the JSON body (range, quartiles, mean, and standard deviation). The goal is to move the calculation to Rust.

The central piece is PyO3, the crate that bridges Rust and Python (the name plays on oxidation: Python trioxide, oxidized Python). The flow Mara lays out is: Flask receives the request, deserializes the JSON, passes the values to the calculation function in Rust, gets the result back, serializes it as JSON, and returns it in the HTTP response.

On the Rust side, a new crate (rstats) is created with dependencies on a statistics library (Rust's std is lean, with OS and synchronization primitives) and on PyO3 itself, enabling the extension-module feature, Python terminology for a callable C library. In Cargo.toml, you need to tell the compiler to generate a C dynamic library (cdylib), instead of the standard Rust artifact, which is only callable by the same compiler version on the same hardware:

[lib]
name = "rstats"
crate-type = ["cdylib"]

[dependencies]
pyo3 = { version = "...", features = ["extension-module"] }

In the Rust code, PyO3 attributes do the heavy lifting of generating wrappers: #[pymodule] turns the Rust module into an importable Python module, and #[pyfunction] generates a wrapper function (for example compute_stats) that, on the Python side, is called like any normal function and under the hood triggers the Rust code. The module name needs to match the crate name, and it will be the name Python imports. For the return data, a struct is created to carry the statistical properties back to Python.

The build uses Maturin, a tool from PyO3's own developers, which packages the crate as a Python module. Once that's done, import rstats in the Flask code works without error. Mara highlights how little code is needed to get a Rust module importable from Python, a point that lowers the entry barrier for anyone who's never done binding between languages.

What this changes for those who build software here

For the Brazilian developer maintaining a Python, Ruby, or Node monolith in production, the takeaway is practical: you can tackle bottleneck after bottleneck without stopping the pipeline or rewriting the system. The question that guides the decision isn't "which language is faster," but "where is my CPU time concentrated in the aggregate," and the natural target is that hot function that runs on every request.

It's worth measuring before touching anything: identify the hotspot with profiling, confirm that the gain justifies the extra cost of deployment and dev environment, and treat legacy code with respect, it carries fixes that a naive rewrite throws away. Useful references cited in the talk are the free book The Rust Programming Language, by Carol Nichols and Steve Klabnik, and Mara's own Refactoring to Rust. The full recording (49min47) is available on InfoQ.