jcs.org
qemu with hax to log dma reads & writes
jcs.org/2018/11/12/vfio
1= Fuzzing =
2
3== Introduction ==
4
5This document describes the virtual-device fuzzing infrastructure in QEMU and
6how to use it to implement additional fuzzers.
7
8== Basics ==
9
10Fuzzing operates by passing inputs to an entry point/target function. The
11fuzzer tracks the code coverage triggered by the input. Based on these
12findings, the fuzzer mutates the input and repeats the fuzzing.
13
14To fuzz QEMU, we rely on libfuzzer. Unlike other fuzzers such as AFL, libfuzzer
15is an _in-process_ fuzzer. For the developer, this means that it is their
16responsibility to ensure that state is reset between fuzzing-runs.
17
18== Building the fuzzers ==
19
20NOTE: If possible, build a 32-bit binary. When forking, the 32-bit fuzzer is
21much faster, since the page-map has a smaller size. This is due to the fact that
22AddressSanitizer mmaps ~20TB of memory, as part of its detection. This results
23in a large page-map, and a much slower fork().
24
25To build the fuzzers, install a recent version of clang:
26Configure with (substitute the clang binaries with the version you installed):
27
28 CC=clang-8 CXX=clang++-8 /path/to/configure --enable-fuzzing
29
30Fuzz targets are built similarly to system/softmmu:
31
32 make i386-softmmu/fuzz
33
34This builds ./i386-softmmu/qemu-fuzz-i386
35
36The first option to this command is: --fuzz_taget=FUZZ_NAME
37To list all of the available fuzzers run qemu-fuzz-i386 with no arguments.
38
39eg:
40 ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=virtio-net-fork-fuzz
41
42Internally, libfuzzer parses all arguments that do not begin with "--".
43Information about these is available by passing -help=1
44
45Now the only thing left to do is wait for the fuzzer to trigger potential
46crashes.
47
48== Adding a new fuzzer ==
49Coverage over virtual devices can be improved by adding additional fuzzers.
50Fuzzers are kept in tests/qtest/fuzz/ and should be added to
51tests/qtest/fuzz/Makefile.include
52
53Fuzzers can rely on both qtest and libqos to communicate with virtual devices.
54
551. Create a new source file. For example ``tests/qtest/fuzz/foo-device-fuzz.c``.
56
572. Write the fuzzing code using the libqtest/libqos API. See existing fuzzers
58for reference.
59
603. Register the fuzzer in ``tests/fuzz/Makefile.include`` by appending the
61corresponding object to fuzz-obj-y
62
63Fuzzers can be more-or-less thought of as special qtest programs which can
64modify the qtest commands and/or qtest command arguments based on inputs
65provided by libfuzzer. Libfuzzer passes a byte array and length. Commonly the
66fuzzer loops over the byte-array interpreting it as a list of qtest commands,
67addresses, or values.
68
69= Implementation Details =
70
71== The Fuzzer's Lifecycle ==
72
73The fuzzer has two entrypoints that libfuzzer calls. libfuzzer provides it's
74own main(), which performs some setup, and calls the entrypoints:
75
76LLVMFuzzerInitialize: called prior to fuzzing. Used to initialize all of the
77necessary state
78
79LLVMFuzzerTestOneInput: called for each fuzzing run. Processes the input and
80resets the state at the end of each run.
81
82In more detail:
83
84LLVMFuzzerInitialize parses the arguments to the fuzzer (must start with two
85dashes, so they are ignored by libfuzzer main()). Currently, the arguments
86select the fuzz target. Then, the qtest client is initialized. If the target
87requires qos, qgraph is set up and the QOM/LIBQOS modules are initialized.
88Then the QGraph is walked and the QEMU cmd_line is determined and saved.
89
90After this, the vl.c:qemu__main is called to set up the guest. There are
91target-specific hooks that can be called before and after qemu_main, for
92additional setup(e.g. PCI setup, or VM snapshotting).
93
94LLVMFuzzerTestOneInput: Uses qtest/qos functions to act based on the fuzz
95input. It is also responsible for manually calling the main loop/main_loop_wait
96to ensure that bottom halves are executed and any cleanup required before the
97next input.
98
99Since the same process is reused for many fuzzing runs, QEMU state needs to
100be reset at the end of each run. There are currently two implemented
101options for resetting state:
1021. Reboot the guest between runs.
103 Pros: Straightforward and fast for simple fuzz targets.
104 Cons: Depending on the device, does not reset all device state. If the
105 device requires some initialization prior to being ready for fuzzing
106 (common for QOS-based targets), this initialization needs to be done after
107 each reboot.
108 Example target: i440fx-qtest-reboot-fuzz
1092. Run each test case in a separate forked process and copy the coverage
110 information back to the parent. This is fairly similar to AFL's "deferred"
111 fork-server mode [3]
112 Pros: Relatively fast. Devices only need to be initialized once. No need
113 to do slow reboots or vmloads.
114 Cons: Not officially supported by libfuzzer. Does not work well for devices
115 that rely on dedicated threads.
116 Example target: virtio-net-fork-fuzz