Mar 29, 2022 | Georgios Konstantopoulos
Foundry is a portable, fast, and modular toolkit for Ethereum application development, written in Rust. Our goal with Foundry is to create the best developer experience for building secure smart contracts on EVM chains. Most importantly: Foundry is built by Solidity developers for Solidity developers.
We announced Foundry v0.1.0 in December 2021 as a first step towards improving the Ethereum development experience. This initial version centered on four key features:
The results since December have convinced us that Foundry solves some major development pain points. On Solmate for instance, it took down testing time from 393s to 2.8s, a 140x improvement. Developers from top protocols like Optimism and MakerDAO are using Foundry as a standalone framework, or will introduce it incrementally in their codebases to improve their build and test process.
Now we're excited to announce Foundry v0.2.0, a major step forward for the toolkit.
TL;DR: we shipped a lot of code over the last three months.
More than 400 pull requests and over 90 contributors later, we are proud to present the results:
foundry.toml so that you can specify project-specific settings without needing to write custom scripts for passing CLI parameters.
We also now have the Foundry Book, our official documentation! While it's still at its early stages, it shows how to set up a Foundry project, write unit & fuzz tests, use cheatcodes, fork from a live network, and more.
In this post we'll focus on the new features that were added to Forge, Foundry's testing framework. Cast, the Swiss Army knife for interacting with a live network, will be covered in a separate post.
If all this sounds exciting, let's dive in.
Foundry devs and users are obsessed with speed. We want our code to compile fast, and the tests to run even faster. As a result, we've been carefully optimizing every part of the compilation and testing pipeline. But first, let's see how fast it really is.
We tested the time it takes to build a project with various caching scenarios. We chose to use openzeppelin-contracts and Uniswap/v3-core as our benchmark repositories. The following scenarios were checked:
We provide the following graphs which plot the time it took to compile for each scenario (lower is better):
Takeaway: Forge compilation is consistently faster by a factor of 1.7-11.3x, depending on the amount of caching involved.
The above results used the same compiler settings but did not attempt to modify Hardhat's settings from the defaults. It is possible that performance can be improved by tuning the settings (e.g. disable plugins or Typescript typechecks) but we chose to benchmark against the default settings since that is what most users experience.
We then proceeded to benchmark performance against other frameworks.
We started by comparing performance with
dapp, Forge's "parent" which recently passed on the torch to Foundry. Thank you DappTools!
(Note that in the above benchmarks compilation was always skipped.)
Then, we ported some tests from
@uniswap/v3-periphery to use Forge, so that we could compare against Hardhat. Deploying Uniswap and running the "exact input" tests took 10s on Hardhat, and 600ms on Forge, a 16x improvement.
Finally, we wanted to test performance when forking against a live network. So we used Convex Finance's shutdown functionality as a benchmark to compare not only against other frameworks, but against hosted services like Blocknative and Tenderly as well.
Check the repository for more context on how the measurements were made.
Takeaway: Forge is the fastest EVM test runner that exists, even beating hosted services in transaction simulation speeds.
How is this all possible? That is a topic for another post dear reader, as each improvement we did deserves its own post.
As a reminder, at the December launch we already supported Hardhat's
console.sol-style logging, and Dapptools'
emit log-based logging which allow you to print values along your contract's execution. We also supported showing the gas cost of each test.
That, however, was not enough! We cannot expect a developer to annotate all their code with logs to figure out what is happening. They also want more granularity on the gas cost of each individual call, instead of the entire test cost.
To provide more runtime introspection, we introduced call traces, which provide a structured output of every external call made by any contract, along with its arguments and return value.
This is the output you'd get when running
forge test -vvvv:
Notice how successful calls are colored in green, reverts are red, and cheatcode calls are blue! More details on how calltraces work can be found in the book.
However, call traces do not give you insight on the opcode by opcode execution of your code, which you may want to view when you are doing low level optimizations or trying to debug a test and need to inspect the state of the EVM's stack and memory.
To solve that, we provide an interactive debugger as part of
forge test --debug and
forge run. Here's how it looks like (you can navigate with keyboard buttons shown at the bottom of the screen):
Most Ethereum developers are familiar with the gas table produced by eth-gas-reporter. We also ported that to work with Foundry. This is how
forge test --gas-report looks like:
We all know that smart contract security is hard. We want to make that easier, by providing functionality so that developers can test every path in their code.
Foundry and DappTools allows overriding the state of the EVM, with a method called "cheatcodes". For example, you can use the
roll cheatcode to set the block number, the
store cheatcode to set an arbitrary storage slot to any value,
prank to impersonate an arbitrary address, among others. This is quite powerful!
We have introduced a variety of new cheatcodes so that developers can have even more control over their tests' state. See all supported cheatcodes below:
Foundry not only lets you write blazing fast configurable unit tests, it provides great UX for writing property-based tests, which we call fuzz tests.
We made two big improvements to the fuzzer:
UINT256_MAX - 1 etc.)
The result? A fuzzer that can find even the most bespoke edge cases very fast. We invite you to try it out, and in the future will be benchmarking it against other industry fuzzers.
The most important phase when attracting a new user, is the installation phase.
Previously, installing Foundry could only be done from source and required having Rust installed. That's not good enough! So we fixed it in two ways:
Installing Foundry now is as simple as seen below:
cast are now available for you to use!
Notably, this works across all platforms, requires no extra tools installed like NPM or Cargo. We believe this is a game changing UX for onboarding on a new tool.
For more information on how foundryup works, check the docs.
When you create a new project via
forge init, a
foundry.toml file is touched at your project's root which defines basic parameters of your project like where are your contracts, your output artifacts and your libraries.
Foundry.toml is very powerful! It lets you configure every aspect of your project, as seen below, instead of you having to pass them each time as a CLI parameter.
Perhaps the most exciting feature of
foundry.toml is "profiles", which let you run different "bundles" of settings depending on a single CLI parameter. As an example, when testing locally you may want to have the default fuzzer runs, whereas on CI you'd want 100000 runs. You'd do this by calling
FOUNDRY_PROFILE=ci forge test on CI, which would use the fuzzer runs from the
ci section of the above
Read more about how to configure your
foundry.toml in the docs.
Foundry is still in the early stages of development. While our results so far are encouraging, we find there's multiple areas for improvement:
A few options:
If none of the above make sense for you, tell us what's missing for you to make the leap, and we'll add it!
First and foremost, we are building Foundry because we needed better tools for internal development than were available publicly. We are proud to be builders and want our tooling to be working for us, not against us. Because we use Foundry for all our projects at Paradigm, we keep a tight feedback loop around what we are building.
Second, by building Foundry, we can help projects in Paradigm’s portfolio ship faster and more safely. Our research and engineering teams spend considerable time with developers in our portfolio, but their time is, unfortunately, finite. By shipping a high-quality development framework, we can scale our efforts and help more teams.
Finally, we hope Foundry can be a contribution of enduring value to Ethereum and the ecosystem that has gathered around it. We encourage everyone building developer tooling to dig into the engineering innovations we have implemented in Foundry and integrate them into their services so more people can benefit from them.
We are proud to have >90 contributors to Foundry. I'd like to highlight the people that have helped us the most in this:
solc Linux aarch64 binaries for us.
foundryup daily and the Foundry/Foundry Support chatrooms who keep the feedback loop tight.
Disclaimer: This post is for general information purposes only. It does not constitute investment advice or a recommendation or solicitation to buy or sell any investment and should not be used in the evaluation of the merits of making any investment decision. It should not be relied upon for accounting, legal or tax advice or investment recommendations. This post reflects the current opinions of the authors and is not made on behalf of Paradigm or its affiliates and does not necessarily reflect the opinions of Paradigm, its affiliates or individuals associated with Paradigm. The opinions reflected herein are subject to change without being updated.