Building a resilient automation engine with Rust

Automation is easy when it’s just one thing happening once. It gets exponentially harder when you’re managing thousands of concurrent stateful agents across a distributed environment. At Olsa, we realized early on that our legacy Node.js execution engine was hitting the ceiling.

We needed something that gave us bare-metal performance with high-level safety guarantees. Enter Rust.

The challenge: state vs scale

Our engine doesn’t just trigger APIs. It maintains long-running state for complex workflows that might wait for human approval for 72 hours or retry a flaky endpoint for 10 minutes.

• Memory safety without a garbage collector
• Fearless concurrency for agent execution
• Small binary footprint for edge deployment
• Strong typing to prevent runtime workflow failures

Architecture overview

“We didn’t just want a faster engine. We wanted an engine that would tell us it was going to fail before we even compiled the first line of code.”

The new engine is a single scheduler binary in front of a sharded state store. Workflows compile to state machines; every await point is a durable checkpoint, so a crashed node resumes exactly where it left off instead of replaying side effects.

Olsa’s new event-loop architecture built on Tokio.

Implementation: the Tokio loop

By leveraging the Tokio runtime, we were able to build a multi-threaded scheduler that handles agent context-switching with near-zero overhead. Here’s a simplified look at how we handle asynchronous agent polling:

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let mut scheduler = WorkflowScheduler::new();

    // Spawn 1,000,000 agents
    for id in 0..1_000_000 {
        tokio::spawn(async move {
            run_agent(id).await;
        });
    }

    Ok(())
}

Performance results

The rewrite took our p99 task latency from 180ms to 9ms and cut the fleet’s memory footprint by two thirds. More importantly, an entire class of production incident — the stuck workflow that silently drops state — became unrepresentable.

What’s next

We’re extending the same state-machine compiler to run agents at the edge, closer to the tools they orchestrate. If making distributed systems boring sounds like fun, we’re hiring.