# Fuzzing#

Fuzzing is a technique for testing software that involves providing invalid, unexpected, or random data as inputs to a computer program.

## Introduction#

The Woke testing framework provides a FuzzTest class that can be used to write fuzz tests. A FuzzTest can be run using the run method with two required arguments:

class CounterTest(FuzzTest):
...

CounterTest().run(sequences_count=10, flows_count=100)


The first argument specifies the number of test sequences to be executed. A sequence is an independent test case - all connected chains are reset after each sequence. Each sequence consists of a given number of flows. A flow is an atomic test step that is executed in a test sequence.

The FuzzTest class provides two properties, sequence_num and flow_num, that can be used to obtain the current sequence and flow numbers, both starting from 0.

### Flows#

A flow is a single test step that is executed in a test sequence. Flows are defined using the @flow decorator:

@flow(precondition=lambda self: self.count > 0)
def flow_decrement(self) -> None:
self.counter.decrement(from_=random_account())
self.count -= 1


Flow functions must be defined inside a test class that inherits from FuzzTest.

The @flow decorator accepts the following keyword arguments:

Argument Description
weight weight defining probability of the flow being executed in a test sequence; defaults to 100
max_times maximum number of times the flow can be executed in a test sequence; defaults to None
precondition function that accepts a single argument self and returns a boolean value; the flow is executed only if the precondition is True

How flow weights work

If a flow has a weight of 100 and another flow has a weight of 50, the first flow will be executed twice as often as the second flow.

@flow(weight=100)
def flow_1(self) -> None:
...

@flow(weight=50)
def flow_2(self) -> None:
...


That means that the probability of flow_1 being executed is 100 / (100 + 50) = 2/3 and the probability of flow_2 being executed is 50 / (100 + 50) = 1/3.

### Invariants#

An invariant is a test that is executed after each flow in a test sequence. Invariants are defined using the @invariant decorator:

@invariant(period=10)
def invariant_count(self) -> None:
assert self.counter.count() == self.count


An optional period argument can be passed to the @invariant decorator. If specified, the invariant is executed only after every period flows.

### Execution hooks#

Execution hooks are functions that are executed during the FuzzTest lifecycle. This is the list of all available execution hooks:

• pre_sequence(self) - executed before each test sequence
• pre_flow(self, flow: Callable) - executed before each flow, accepts the flow function to be executed as an argument
• post_flow(self, flow: Callable) - executed after each flow, accepts the flow function that was executed as an argument
• pre_invariants(self) - executed before each set of invariants
• pre_invariant(self, invariant: Callable) - executed before each invariant, accepts the invariant function to be executed as an argument
• post_invariant(self, invariant: Callable) - executed after each invariant, accepts the invariant function that was executed as an argument
• post_invariants(self) - executed after each set of invariants
• post_sequence(self) - executed after each test sequence

The whole FuzzTest lifecycle is visualized in the following diagram:

### Example#

Putting all of the above together, here is an example of a FuzzTest that tests the Counter contract:

from woke.testing import *
from woke.testing.fuzzing import *
from pytypes.contracts.Counter import Counter

class CounterTest(FuzzTest):
counter: Counter
count: int

def pre_sequence(self) -> None:
self.counter = Counter.deploy()
self.count = 0

@flow()
def flow_increment(self) -> None:
self.counter.increment()
self.count += 1

@flow()
def flow_decrement(self) -> None:
with may_revert(Panic(PanicCodeEnum.UNDERFLOW_OVERFLOW)) as e:
self.counter.decrement()

if e.value is None:
self.count -= 1
else:
assert self.count == 0

@invariant(period=10)
def invariant_count(self) -> None:
assert self.counter.count() == self.count

@default_chain.connect()
def test_counter():
default_chain.set_default_accounts(default_chain.accounts[0])
CounterTest().run(sequences_count=30, flows_count=100)


The test performs 30 test sequences, each consisting of 100 flows. It tests with two flows of the same probability: flow_increment and flow_decrement. The invariant invariant_count is executed after every 10 flows.

## Generating random data#

There are two ways to generate random data in Woke fuzz tests.

### Flow arguments#

Every flow function can accept additional arguments to the implicit self. These arguments are generated based on the type hints:

@flow()
def flow_set_count(self, count: uint) -> None:
self.counter.set_count(count, from_=self.counter.owner())
self.count = count


Flow argument types can be any of the following:

• integer types ranging from uint8 to uint256 and from int8 to int256, including uint and int,
• byte types ranging from bytes1 to bytes32, including bytes and bytearray,
• List, including List1 to List32 helper annotations (e.g. List16[uint8]),
• bool,
• str,
• Address, does never generate the zero address,
• any Enum, including enums generated in pytypes,
• any dataclass, including dataclasses generated in pytypes.

All flow arguments are generated non-biased, i.e. the probability of generating a value of a given type is the same for all values of that type. For types that have length, the length is generated in the range 0 to 64.

For generating fine-tuned random data, it is recommended to use the random functions from the woke.testing.fuzzing module.

### Random functions#

Woke testing framework provides a set of random functions that can be used to generate random data.

#### random_account()#

random_account() returns a random account from a given chain. It accepts the following keyword arguments:

Argument Description Default value
lower_bound lower bound index of chain.accounts to choose from 0
upper_bound upper bound index of chain.accounts to choose from None (i.e. len(chain.accounts))
predicate predicate that the account must satisfy None (i.e. no predicate)
chain chain to choose the account from default_chain

random_address() returns a random address. It accepts the following keyword arguments:

Argument Description Default value
zero_address_prob probability of generating the zero address 0

#### random_int(min, max)#

random_int(min, max) returns a random integer in the range min to max. It accepts the following keyword arguments:

Argument Description Default value
min_prob probability of generating min None (i.e. 1 / (max - min + 1))
max_prob probability of generating max None (i.e. 1 / (max - min + 1))
zero_prob probability of generating 0, if min < 0 < max None (i.e. 1 / (max - min + 1))
edge_values_prob value to use for min_prob, max_prob and
zero_prob if not set
None

#### random_bool()#

random_bool() returns a random boolean value. It accepts the following keyword arguments:

Argument Description Default value
true_prob probability of generating True 0.5

#### random_string(min, max)#

random_string(min, max) returns a random string of length in the range min to max. It accepts the following keyword arguments:

Argument Description Default value
alphabet alphabet to choose characters from string.printable
predicate predicate that the string must satisfy None (i.e. no predicate)

#### random_bytes(min, max)#

random_bytes(min, max) returns a random byte array of length in the range min to max. If max is not specified, it generates exactly min bytes. It accepts the following keyword arguments:

Argument Description Default value
predicate predicate that the byte array must satisfy None (i.e. no predicate)

## Launching tests in parallel#

Woke testing framework allows running the same test in parallel with different random seeds. Multiprocess tests are launched using the woke fuzz command:

woke fuzz tests/test_counter_fuzz.py -n 5


Info

The command woke fuzz does not utilize the pytest framework to collect and execute tests. As a consequence, the pytest features like fixtures are not available. Test functions must start with the test prefix. Test classes are not supported.

If a test process encounters an error, the user is prompted whether to debug the test or continue fuzzing. While debugging, other processes are still running in the background.

By default, nothing but status of each test is printed to the console. Using the --passive flag, the output of the first process is printed to the console. Standard output and standard error of all processes are redirected to the .woke-logs/fuzz directory.

Reproducing a failed test

For every process, Woke generates a random seed that is used to initialize the random number generator. The seed is printed to the console and can be used to reproduce the test failure:

woke fuzz tests/test_counter_fuzz.py -n 5 -s 62061e838798ad0f


A random seed can be specified using the -s flag. Multiple -s flags are allowed.