May 03, 2024 | Georgios Konstantopoulos
Contents
Reth is an all-in-one toolkit for building high performance and customizable nodes. We recently published our performance roadmap for improving Reth’s performance >100x, and Reth AlphaNet, our testnet rollup for pushing Reth’s modularity and extensibility to the limits.
Today, we are excited to announce Reth Execution Extensions (ExEx). ExEx is a framework for building performant and complex off-chain infrastructure as post-execution hooks. Reth ExExes can be used to implement rollups, indexers, MEV bots and more with >10x less code than existing methods. With this release, we demonstrate from scratch a prod-ready reorg tracker in <20 LoC, an indexer in <250 LoC and a rollup in <1000 LoC.
ExEx was co-architected with init4, a research collective building next-generation Ethereum infrastructure. We look forward to continuing to collaborate with the init4 team as we make Reth the #1 platform for building crypto infrastructure!
A blockchain is a clock that confirms blocks with transaction data on a regular interval. Off-chain infrastructure subscribes to these regular block updates and updates its own internal state as a response.
For example, consider how an Ethereum indexer works:
eth_subscribe
or by polling with eth_getFilterChanges.
This is the standard Extract Transform Load (ETL) pattern that you see in large scale data pipelines, with companies like Fivetran owning the data extraction, Snowflake handling the loading into a data warehouse, and customers focusing on writing the transformation’s business logic.
We observe that this same pattern also applies to other pieces of crypto infrastructure such as rollups, MEV searchers, or more complex data infrastructure like Shadow Logs.
Using that as motivation, we identify key challenges when building ETL pipelines for Ethereum nodes:
There is a need for a better API for building off-chain infrastructure that depends on a node’s state changes. That API must be performant, ‘batteries-included’ and reorg-aware. We need an Airflow moment for building Ethereum ETL infrastructure and job orchestration.
Execution Extensions (ExExes) are post-execution hooks for building real-time, high performance and zero-operations off-chain infrastructure on top of Reth.
An Execution Extension is a task that derives its state from Reth's state. Some examples of such state derives are rollups, indexers, MEV extractors, and more. We expect that developers will build reusable ExExes that compose with each other in a standardized way, similar to how Cosmos SDK modules or Substrate Pallets work.
We co-architected Execution Extensions with the init4 team (follow them!), a new research collective building next-generation Ethereum infrastructure. We are excited to continue collaborating with their team as we productionize ExExes and make Reth the #1 platform for building crypto infrastructure!
We are still early in the best practices of building ExExes, and we’d like to invite developers to join us in exploring this new frontier of building off-chain crypto infrastructure. Please reach out with ideas to collaborate.
In Rust terms, an ExEx is a Future that is run indefinitely alongside Reth. ExExes are initialized using an async closure that resolves to the ExEx. Here is the expected end to end flow:
ExExNotification
which includes a list of blocks committed to the chain, and all associated transactions & receipts, state changes and trie updates with them.async
functions that derive state such as a rollup block. The stream exposes a ChainCommitted variant for appending to ExEx state and a ChainReverted/Reorged-variant for undoing any changes. This is what allows ExExes to be operating at native block time, while also exposing a sane API for handling reorgs safely, instead of not handling reorgs and introducing latency.ExExManager
that is responsible for routing notifications from Reth to ExExes and ExEx events back to Reth, while Reth’s task executor drives ExExes to completion.install_exex
API of the Node Builder.Here is how this roughly looks like from the node developer’s perspective:
use futures::Future;
use reth_exex::{ExExContext, ExExEvent, ExExNotification};
use reth_node_api::FullNodeComponents;
use reth_node_ethereum::EthereumNode;
// The `ExExContext` is available to every ExEx to interface with the rest of the node.
//
// pub struct ExExContext<Node: FullNodeComponents> {
// /// The configured provider to interact with the blockchain.
// pub provider: Node::Provider,
// /// The task executor of the node.
// pub task_executor: TaskExecutor,
// /// The transaction pool of the node.
// pub pool: Node::Pool,
// /// Channel to receive [`ExExNotification`]s.
// pub notifications: Receiver<ExExNotification>,
// // .. other useful context fields
// }
async fn exex<Node: FullNodeComponents>(mut ctx: ExExContext<Node>) -> eyre::Result<()> {
while let Some(notification) = ctx.notifications.recv().await {
match ¬ification {
ExExNotification::ChainCommitted { new } => {
// do something
}
ExExNotification::ChainReorged { old, new } => {
// do something
}
ExExNotification::ChainReverted { old } => {
// do something
}
};
}
Ok(())
}
fn main() -> eyre::Result<()> {
reth::cli::Cli::parse_args().run(|builder, _| async move {
let handle = builder
.node(EthereumNode::default())
.install_exex("Minimal", |ctx| async move { exex(ctx) } )
.launch()
.await?;
handle.wait_for_node_exit().await
})
}
The above <50 LoC snippet encapsulates defining and installing an ExEx. It is extremely powerful and allows extending your Ethereum node’s functionality with zero additional pieces of infrastructure.
Let’s walk through some examples now.
The “Hello World” of Execution Extensions is a reorg tracker. The ExEx shown in the screenshot below illustrates logging whether there was a new chain or a reorganization. One could build a reorg tracker on top of their Reth node easily just by parsing the info logs emitted by the below ExEx.
In this example, the old
and new
chains have full access to every state change in that range of blocks, along with the trie updates and other useful information in the Chain
struct.
async fn exex<Node: FullNodeComponents>(mut ctx: ExExContext<Node>) -> eyre::Result<()> {
while let Some(notification) = ctx.notifications.recv().await {
match ¬ification {
ExExNotification::ChainCommitted { new } => {
info!(committed_chain = ?new.range(), "Received commit");
}
ExExNotification::ChainReorged { old, new } => {
info!(from_chain = ?old.range(), to_chain = ?new.range(), "Received reorg");
}
ExExNotification::ChainReverted { old } => {
info!(reverted_chain = ?old.range(), "Received revert");
}
};
if let Some(committed_chain) = notification.committed_chain() {
ctx.events.send(ExExEvent::FinishedHeight(committed_chain.tip().number))?;
}
}
Ok(())
}
Now that we have the basics of hooking on node events, let's build a more elaborate example, such as an indexer for deposits and withdrawals in common OP Stack chains, using SQlite as the backend.
In this case:
sol!
macro to generate type-safe ABI decoders (this is an extremely powerful macro that we encourage developers to dive deeper in).ExExNotification
we proceed to read the logs for every committed block, decode it, and then insert it into SQLite.ExExNotification
is for a chain reorganization, then we remove the corresponding entries from the SQLite tables.That's it! Super simple, and probably the highest performance locally hosted real-time indexer you can build in 30 minutes. See the code below, and go through the full example.
use alloy_sol_types::{sol, SolEventInterface};
use futures::Future;
use reth_exex::{ExExContext, ExExEvent};
use reth_node_api::FullNodeComponents;
use reth_node_ethereum::EthereumNode;
use reth_primitives::{Log, SealedBlockWithSenders, TransactionSigned};
use reth_provider::Chain;
use reth_tracing::tracing::info;
use rusqlite::Connection;
sol!(L1StandardBridge, "l1_standard_bridge_abi.json");
use crate::L1StandardBridge::{ETHBridgeFinalized, ETHBridgeInitiated, L1StandardBridgeEvents};
fn create_tables(connection: &mut Connection) -> rusqlite::Result<()> {
connection.execute(
r#"
CREATE TABLE IF NOT EXISTS deposits (
id INTEGER PRIMARY KEY,
block_number INTEGER NOT NULL,
tx_hash TEXT NOT NULL UNIQUE,
contract_address TEXT NOT NULL,
"from" TEXT NOT NULL,
"to" TEXT NOT NULL,
amount TEXT NOT NULL
);
"#,
(),
)?;
// .. rest of db initialization
Ok(())
}
/// An example of ExEx that listens to ETH bridging events from OP Stack chains
/// and stores deposits and withdrawals in a SQLite database.
async fn op_bridge_exex<Node: FullNodeComponents>(
mut ctx: ExExContext<Node>,
connection: Connection,
) -> eyre::Result<()> {
// Process all new chain state notifications
while let Some(notification) = ctx.notifications.recv().await {
// Revert all deposits and withdrawals
if let Some(reverted_chain) = notification.reverted_chain() {
// ..
}
// Insert all new deposits and withdrawals
if let Some(committed_chain) = notification.committed_chain() {
// ..
}
}
Ok(())
}
/// Decode chain of blocks into a flattened list of receipt logs, and filter only
/// [L1StandardBridgeEvents].
fn decode_chain_into_events(
chain: &Chain,
) -> impl Iterator<Item = (&SealedBlockWithSenders, &TransactionSigned, &Log, L1StandardBridgeEvents)>
{
chain
// Get all blocks and receipts
.blocks_and_receipts()
// .. proceed with decoding them
}
fn main() -> eyre::Result<()> {
reth::cli::Cli::parse_args().run(|builder, _| async move {
let handle = builder
.node(EthereumNode::default())
.install_exex("OPBridge", |ctx| async move {
let connection = Connection::open("op_bridge.db")?;
create_tables(&mut connection)?;
Ok(op_bridge_exex(ctx, connection))
})
.launch()
.await?;
handle.wait_for_node_exit().await
})
}
Let's do something more interesting now, and build a minimal Rollup as an ExEx, with an EVM runtime and SQLite as the backend!
If you zoom out, even Rollups are ETL-ish pipelines:
In this example, we demonstrate a simplified rollup that derives its state from RLP encoded EVM transactions posted to Zenith (a Holesky smart contract for posting our rollup's block commitments) driven by a simple block builder, both built by the init4 team.
The example specifically:
revm
Database traits to use SQLite as an EVM backend.Again, super simple. It also works with blobs!
ExEx Rollups are extremely powerful because we can now run any number of rollups on Reth without additional infrastructure, by installing them as ExExes.
We are working on extending the example with blobs, and providing a built-in sequencer, for a more complete end to end demo. Reach out if this is something you'd like to build, as we think this has potential for introducing L2 PBS, decentralized / shared sequencers or even SGX-based sequencers and more.
Example snippets below.
use alloy_rlp::Decodable;
use alloy_sol_types::{sol, SolEventInterface, SolInterface};
use db::Database;
use eyre::OptionExt;
use once_cell::sync::Lazy;
use reth_exex::{ExExContext, ExExEvent};
use reth_interfaces::executor::BlockValidationError;
use reth_node_api::{ConfigureEvm, ConfigureEvmEnv, FullNodeComponents};
use reth_node_ethereum::{EthEvmConfig, EthereumNode};
use reth_primitives::{
address, constants,
revm::env::fill_tx_env,
revm_primitives::{CfgEnvWithHandlerCfg, EVMError, ExecutionResult, ResultAndState},
Address, Block, BlockWithSenders, Bytes, ChainSpec, ChainSpecBuilder, Genesis, Hardfork,
Header, Receipt, SealedBlockWithSenders, TransactionSigned, U256,
};
use reth_provider::Chain;
use reth_revm::{
db::{states::bundle_state::BundleRetention, BundleState},
DatabaseCommit, StateBuilder,
};
use reth_tracing::tracing::{debug, error, info};
use rusqlite::Connection;
use std::sync::Arc;
mod db;
sol!(RollupContract, "rollup_abi.json");
use RollupContrac:{RollupContractCalls, RollupContractEvents};
const DATABASE_PATH: &str = "rollup.db";
const ROLLUP_CONTRACT_ADDRESS: Address = address!("74ae65DF20cB0e3BF8c022051d0Cdd79cc60890C");
const ROLLUP_SUBMITTER_ADDRESS: Address = address!("B01042Db06b04d3677564222010DF5Bd09C5A947");
const CHAIN_ID: u64 = 17001;
static CHAIN_SPEC: Lazy<Arc<ChainSpec>> = Lazy::new(|| {
Arc::new(
ChainSpecBuilder::default()
.chain(CHAIN_ID.into())
.genesis(Genesis::clique_genesis(CHAIN_ID, ROLLUP_SUBMITTER_ADDRESS))
.shanghai_activated()
.build(),
)
});
struct Rollup<Node: FullNodeComponents> {
ctx: ExExContext<Node>,
db: Database,
}
impl<Node: FullNodeComponents> Rollup<Node> {
fn new(ctx: ExExContext<Node>, connection: Connection) -> eyre::Result<Self> {
let db = Database::new(connection)?;
Ok(Self { ctx, db })
}
async fn start(mut self) -> eyre::Result<()> {
// Process all new chain state notifications
while let Some(notification) = self.ctx.notifications.recv().await {
if let Some(reverted_chain) = notification.reverted_chain() {
self.revert(&reverted_chain)?;
}
if let Some(committed_chain) = notification.committed_chain() {
self.commit(&committed_chain)?;
self.ctx.events.send(ExExEvent::FinishedHeight(committed_chain.tip().number))?;
}
}
Ok(())
}
/// Process a new chain commit.
///
/// This function decodes all transactions to the rollup contract into events, executes the
/// corresponding actions and inserts the results into the database.
fn commit(&mut self, chain: &Chain) -> eyre::Result<()> {
let events = decode_chain_into_rollup_events(chain);
for (_, tx, event) in events {
match event {
// A new block is submitted to the rollup contract.
// The block is executed on top of existing rollup state and committed into the
// database.
RollupContractEvents::BlockSubmitted(_) => {
// ..
}
// A deposit of ETH to the rollup contract. The deposit is added to the recipient's
// balance and committed into the database.
RollupContractEvents::Enter(RollupContract::Enter {
token,
rollupRecipient,
amount,
}) => {
// ..
_ => (),
}
}
Ok(())
}
/// Process a chain revert.
///
/// This function decodes all transactions to the rollup contract into events, reverts the
/// corresponding actions and updates the database.
fn revert(&mut self, chain: &Chain) -> eyre::Result<()> {
let mut events = decode_chain_into_rollup_events(chain);
// Reverse the order of events to start reverting from the tip
events.reverse();
for (_, tx, event) in events {
match event {
// The block is reverted from the database.
RollupContractEvents::BlockSubmitted(_) => {
// ..
}
// The deposit is subtracted from the recipient's balance.
RollupContractEvents::Enter(RollupContract::Enter {
token,
rollupRecipient,
amount,
}) => {
// ..
}
_ => (),
}
}
Ok(())
}
}
fn main() -> eyre::Result<()> {
reth::cli::Cli::parse_args().run(|builder, _| async move {
let handle = builder
.node(EthereumNode::default())
.install_exex("Rollup", move |ctx| async {
let connection = Connection::open(DATABASE_PATH)?;
Ok(Rollup::new(ctx, connection)?.start())
})
.launch()
.await?;
handle.wait_for_node_exit().await
})
}
This question can be reframed as “What can be modeled as a post-execution hook?”. Turns out a lot of things!
We see a few valuable ExExes that should be built:
Currently, ExExes need to be installed on the node with a custom build in your main function. We aspire to make ExExes dynamically loaded as plugins, and expose a Docker Hub-esque reth pull
API, such that developers can distribute their ExExes over the air to node operators easily.
We want to make Reth a platform that provides stability & performance on core node operations, while also being a launchpad for innovation.
The Reth project is hopefully going to change how people think about building high performance off-chain infra, and ExExes are just the beginning. We are excited to continue building infrastructure on Reth, and invest in it.
Reach out to georgios@paradigm.xyz if you want to build an ExEx project, or contribute directly on Github.
Disclaimer: This post is for general information purposes only. It does not constitute investment advice or a recommendation or solicitation to buy or sell any investment and should not be used in the evaluation of the merits of making any investment decision. It should not be relied upon for accounting, legal or tax advice or investment recommendations. This post reflects the current opinions of the authors and is not made on behalf of Paradigm or its affiliates and does not necessarily reflect the opinions of Paradigm, its affiliates or individuals associated with Paradigm. The opinions reflected herein are subject to change without being updated.