Developers need to test the reliability and security of their programs before deployment. Traditional unit tests often fail to reveal the edge-case vulnerabilities. Trdelnik addresses this by introducing fuzz testing for Solana programs, which generates a large set of random inputs and call orders to probe for unexpected weaknesses.
Why we created Trdelnik
Programs for the Solana blockchain are mostly written in Rust programming language. There is a good reason to use Rust because it guarantees memory safety without compromising performance as it is the case with other programming languages that use garbage collection. Rust on the other hand uses a new unique concept of ownership. The downside is that Rust might become very complex and the learning curve is steep.
But wait, have you also heard that “only the best programmers can write in Rust and the best make fewer mistakes”? Well even if it was partially true, there are better ways to prevent bugs and one of them is testing. Extensive and systematic testing is an integral part of any software development and is a great way to discover bugs early during development.
How is it possible that most projects have only basic tests or no tests at all? Believe it or not, writing good tests is not as easy as it might seem and in some cases can take even longer than the development of the product alone. In the fast-paced world of crypto, the timelines are often very short and the pressure to launch new projects is extremely high.
That is why we have developed Trdelnik, our Rust-based testing framework providing several convenient developer tools for testing Solana programs written in Anchor. The main goal of Trdelnik is to simplify the setup of the testing environment, provide an automatically generated API to send instructions to custom programs and accelerate the testing process.
Fuzz Testing
A new feature recently introduced to Trdelnik is fuzz testing. It is an automated software testing technique that provides generated random, invalid, or unexpected input data to your program. This helps to discover unknown bugs and vulnerabilities and may prevent zero-day exploits of your program.
There are several fuzzers used for Rust programs, however, googling for Solana fuzz tests does not show any results and that is why we decided to integrate this feature into our framework. Under the hood, Trdelnik uses a well known fuzzer honggfuzz developed by Google.
In the following text we will describe step by step how to use Trdelnik for fuzz testing.
TL;DR
Fuzzing with Trdelnik Step by Step
In this tutorial we will go through the complete setup involving these steps:
- Setting up a new Anchor project
- Creating a program that contains bugs to detect
- Initializing Trdelnik test framework
- Writing a simple fuzz test
- Running the fuzz test
- Debugging our program using fuzz test crash files
- Setting up a new Anchor project
For the purpose of this tutorial we will create a new Anchor project. If you do not have Anchor framework yet, then install the version 0.28.0.
Open a terminal and go to your project folder where we will create the new project and verify if the project can be built:
Verify that the Anchor project is initialized and built successfully.
Creating a program that contains bugs to detect
Next we will create a simple program where we will intentionally introduce bugs that we will try to find with our fuzzer. Open the source file of your program programs /my-trdelnik-fuzz-test/src/lib.rs and replace everything after the declare_id!() macro with the following code:
It is a simple program with two instructions: initialize and update. The initialize instruction will create the necessary data account and the update instruction will update the on-chain data. It also contains an intentional panic! macro that will immediately terminate the program simulating a crash if the first input to the program equals the constant MAGIC_NUMBER.
Now you can again verify if your program builds successfully using anchor build.
Initialize Trdelnik test framework
In order to use Trdelnik and the fuzzer, we have to install them:
If you had Trdelnik already installed then you need to upgrade to version 0.5.0.
After that, we must initialize Trdelnik framework in our project using the command:
This command will automatically generate the .program_client folder with an API to our program, add necessary dependencies to the configuration files and generate trdelnik tests templates.
Write a simple fuzz test
The fuzz test target template is located in the trdelnik-tests/src/bin/fuzz_target.rs file. This file can be modified according to your needs.
Now, you can only replace its content with the following code:
For the purpose of this tutorial and for simplicity we are fuzzing only the instruction parameters. Namely the two parameters input1 and input2 of the update instruction.
If you look at the fuzz target code, you can see that the main function contains an infinite loop with the fuzz! macro that takes a closure where we pass the FuzzData structure.
Let’s dissect the code a little. The entry point of the fuzz target is the main function, where we call the fuzz! macro over and over again. This macro executes the code in the closure and in each loop the passed fuzz_data are different.
We are using the arbitrary crate that enables us to easily transform unstructured random data to structured data as we defined it in the FuzzData structure. The closure code gets executed and if the program does not crash the fuzz test continues with new loop iteration and the fuzz_data passed to the closure is modified. If the fuzz_data variable contains data that produces a crash in our program, the fuzzer automatically stores the data in a separate fuzz crash file ./trdelnik-tests/hfuzz_workspace/<TARGET_NAME>/<CRASH_FILE_NAME>.fuzz. This is useful especially for subsequent debugging.
The code executed in the closure in our fuzz target first creates a new TestProgram and adds our program to the test environment. Then the test client is started that enables us to send transactions to our program. Having that, we have to first initialize our program. We create the initialize instruction with help of Trdelnik that automatically generated the initialize_ix function for us. Finally we create a new transaction, that we sign and send via the client to the test environment. Great, our program is initialized!
Now we call the fuzz_update_ix function in order to supply random data to the update instruction that we want to fuzz and where we want to find bugs. Again, we construct the update instruction using the automatically generated update_ix function and we supply the randomly generated parameters from the FuzzData structure. Finally, we create the transaction, sign and send it. And that’s it, our fuzz target is ready to go!
Run the fuzz test
Trdelnik provides a convenient way to run the fuzz tests. Anywhere in your Anchor project, you can execute the command:
So in our case if we replace the <TARGET_NAME> with the actual name it gets the following:
Once you execute this command, the whole project has to be built with instrumentation for fuzzing so it takes some time. After the build is finished, the fuzzer starts automatically and looks like this:
At the top, you can see fuzzing statistics. Especially how many times your program crashed, how many of these crashes were unique, how many iterations were done and so on.
To stop fuzzing, you can simply hit CTRL+C. As Trdelnik uses Honggfuzz under the hood, you can also pass parameters directly to the fuzzer using environment variables.
For example:
If you run the command above, the fuzzer will stop after the first encountered crash and passing the -Q flag allows us to see the Solana logs. In our Solana example program, we use the msg! macro to display the value of the update instruction parameters input1 and input2 that you can see from the log that have values 254 and 255 respectively.
Debug our program using fuzz test crash files
As you can see from the log above, the data that caused the crash of our program was stored in the fuzz crash file in ./trdelnik-tests/hfuzz_workspace/fuzz_target folder.
Now it is possible to use this crash file and “replay” the crash in the debugger to inspect the bug. The corresponding command is:
So in our case if we replace the <TARGET_NAME> and <CRASH_FILE_PATH> with the actual values. The name of the crash file might differ so you will have to modify it accordingly.
In our case the resulting command is as follows:
Once you execute this command, the whole project has to be built for debugging. After that, the fuzzer runs your program with supplied parameters from the crash file and simulates the crash again for inspection.
Now you can see in the debugger, that the thread panicked at ‘Black magic not supported!’, programs/my-trdelnik-fuzz-test/src/lib.rs:27:13
This is the expected output because we have intentionally introduced a panic in our program if the input1 variable is equal to our MAGIC_NUMBER which is 254.
While in the debugger, you can either execute a help command for further actions or quit by executing the q command.
If you want to try to find another bug in the program, you can uncomment the if statement
in the buggy_math_function function in the Solana program in programs/my-trdelnik-fuzz-test/src/lib.rs and run the fuzzer again. After some time, a new unique crash should be found that you can again analyze using the debugger as shown before.
Conclusion
We have shown how to use the Trdelnik framework to write fuzz tests for Solana programs written in Anchor. You can find the whole example project in Trdelnik’s GitHub repository.
The goal of Trdelnik is not to implement a new fuzzer but rather to provide a convenient way to use the existing honggfuzz fuzzer without the hassle of setting up the testing environment.
It is as simple as initializing Trdelnik in your project and you are ready to fuzz!
The next steps will be the accounts and instructions flow fuzzing.
Stay tuned for more tutorials in the future!
