The Executor is the component of a fuzzer responsible for running the test file (Testcase) in the software being tested (Target). The ideal objective when designing an Executor is to execute Testcases as quickly and efficiently as possible, without introducing any instability in the Target that could cause false positives or inaccurate code coverage reports. Achieving this requires an initial study of the Target to determine the best strategy, depending on the characteristics of the fuzzer and the goals to be achieved.

In this post, we will discuss two possible strategies. We will choose the JavaScript V8 engine of the Chromium project as the Target. V8 is one of the most powerful JavaScript engines, known for its security, high performance, and numerous features and configurations. You can download and compile it by following the instructions at V8.dev.

We will run our tests on the metrics server to measure the execution capacity of each strategy and explain their pros and cons.

The chosen Testcase is as follows:

This is the most rudimentary method, based on compiling V8 to generate the d8 binary, which accepts JavaScript files via command line arguments for execution (e.g., ./d8 testcase.js). The fuzzer runs the command in a new process for each Testcase.

This approach is slower compared to other techniques because the operating system must launch a new d8 process and initialize the entire V8 environment (including the V8 Isolate Heap, loading of snapshots, V8 Global Templates, V8 Context, etc.) for each Testcase. If our fuzzer uses metrics such as code coverage, a possible communication method would be via files, utilizing ASAN_OPTIONS and its coverage_dir option. If we wanted to create communication between the fuzzer and the JavaScript engine, we could use the print JS API and stdout.

Although this strategy is not the most efficient in terms of speed, it has the advantage of full stability because each execution starts in a completely new and clean environment. Its implementation would be simple, and the fuzzer would have access to all the features that the d8 binary provides, including various execution configurations and APIs not part of V8 but added by V8 engineers, such as setTimeout, Worker, d8.dom.Div, and dynamic and static import/export.

This technique is recommended when the execution time of each Testcase is long, such as when our fuzzer produces Testcases that take more than 20 seconds to complete.

The pseudo-code of the Executor would be:

After running this technique on our server, we achieved a total of 6,967 executions every 300 seconds.

This technique aims to minimize the creation of new processes and the initialization of the V8 environment, significantly improving the runtime of each Testcase.

The idea is to create a new process that contains or initializes everything necessary to execute Testcases in a synchronized loop between the fuzzer. For V8, after initializing the V8 Isolate Heap, we perform multiple executions in a new V8 context for each Testcase. This reduces the number of operations and achieves high stability between executions.

This strategy is faster than the previous one and is ideal for Testcases running within seconds or fractions of a second. However, it requires customizing the Target, which involves more development and maintenance.

The general pseudo-code for this technique would be:

After testing, we achieved a total of 17,670 executions every 300 seconds, nearly tripling the performance of Strategy 1.

We have two options for implementing this technique:

We would use the d8 binary and its support for the Fuzzilli fuzzer and its executor called REPRL, which facilitates Fuzzer-d8 communication via stdin. This allows d8 to run multiple Testcases in a loop, keeping the Target process alive and reusing the same V8 Isolate Heap.

If we use code coverage, we can collect feedback through shared memory established as a global variable (SHM_ID), as d8 overwrites the APIs _sanitizer_cov_trace_pc_guard and _sanitizer_cov_pc_guardinit to report to that memory’s name.

This technique retains all the features and APIs that d8 provides. However, it requires studying the implementation and maintaining updates for potential changes in d8.

This strategy involves compiling V8 as an external library and integrating it into the fuzzer. This approach allows full control over initializing the JavaScript engine and executing the Testcase, ensuring a stable environment with minimal duplication during execution. It also provides access to the entire JavaScript engine and memory, enabling the development of advanced bug discovery techniques.

V8 would be loaded into memory within the fuzzer’s process. Once the V8 Isolate Heap is initialized, the fuzzer forks and runs Testcases in a loop in a separate process, which inherits the necessary initializations from the fork.

We can also create fast communication channels via shared memory to collect code coverage feedback or interact with the JavaScript engine. Operating within the same ecosystem enables precise code coverage techniques, which help create the most stable environment possible.

However, this development requires more time, and we would need to implement the features and APIs that d8 already provides.

Each technique has its pros and cons, depending on the characteristics of the fuzzer. Key considerations include the runtime of each Testcase, the stability required to avoid false positives, the precision of code coverage, and the available development time.