Fuzzlyn

Fuzzlyn is a fuzzer which utilizes Roslyn to generate random C# programs. It runs these programs on .NET core and ensures that they give the same results when compiled in debug and release mode.

We developed Fuzzlyn as a project for the 2018 Language-Based Security course at Aarhus University. Using Fuzzlyn, we have found and reported several bugs in RyuJIT used both by .NET Core and the full .NET framework. We have also found and reported a (harmless) bug in Roslyn.

It has found many thousands of programs producing deviating behavior. Some of the first examples we found can be seen in the examples folder in the v1.0 tag (most of these have since been fixed). To take a couple of them, Fuzzlyn automatically found and produced the following programs:

// Generated by Fuzzlyn on 2018-06-03 16:17:09

// Seed: 10744458083861091494

// Reduced from 21.3 KB to 0.2 KB

// Debug: Outputs '246'

// Release: Outputs '4294967286'

public class Program

{

    static sbyte s_1 = -10;

    public static void Main()

    {

        ulong vr44 = (byte)(0U ^ s_1);

        System.Console.WriteLine(vr44);

    }

}

// Generated by Fuzzlyn on 2018-06-03 16:15:22

// Seed: 10187462581749713401

// Reduced from 186.5 KB to 0.2 KB

// Debug: Runs successfully

// Release: Throws 'System.DivideByZeroException'

public class Program

{

   public static void Main()

   {

       var vr219 = 'N' % ((35815 / M1(new ushort[]{65535})) | 1);

   }



   static ushort M1(ushort[] arg2)

   {

       return arg2[0];

   }

}

Changelog v1.3

• Thread stderr through and change simplification order

• Simplify initializers before constants since that will reduce the number of constants by a lot.

Download

git clone https://github.com/jakobbotsch/Fuzzlyn.git

Using Fuzzlyn to find bugs

To run Fuzzlyn you must specify the number of programs to generate and check. To run a Fuzzlyn instance that generates a million programs and tries to find bugs in these, run:

dotnet fuzzlyn.dll --num-programs 1000000

Note that this only works with .NET core (i.e. a host like dotnet must be running the program). This is because Fuzzlyn runs instances of itself and it uses the dotnet that started Fuzzlyn to do this.

This command will not produce any output to stdout. However, when a program with deviating behavior is found, Fuzzlyn will append its seed and information about its execution to a file in the current directory called Execution_Mismatch.txt.

Regenerating full programs

When a seed has been obtained, the full program can be regenerated by doing the following:

dotnet fuzzlyn.dll --seed <seed here> --output-source

This will generate exactly the code that produced deviating behavior at runtime. However, these examples cannot be run directly because they include checksumming of variables, which requires the IRuntime interface to be passed to the Mainmethod. Instead, you can disable checksumming by passing the –checksum- (note the minus) switch:

dotnet fuzzlyn.dll --seed <seed here> --output-source --checksum-

These examples are not very useful because they are very big. For that reason, Fuzzlyn includes an automatic reducer, which takes a seed and reduces the program specified by that seed to something smaller, while retaining the interesting behavior.

Reducing programs

To reduce a program, use the –reduce switch:

dotnet fuzzlyn.dll –seed <seed here> –reduce

Depending on the size of the program, this will take a while (the biggest example in the examples directory takes roughly 10 minutes to reduce on an i7-4770k). However, most programs do not take longer than a couple of minutes. The output of this command will be a small C# program, that includes information about its seed, size and runtime behavior.

More…

Copyright (c) 2018 Jakob Botsch Nielsen
Source: https://github.com/jakobbotsch/