Property

Using Hedgehog, the programmer writes assertions about logical properties that a function should fulfill.

Take List.rev as an example, which is a function that returns a new list with the elements in reverse order:

List.rev [1; 2; 3];;

val it : int list = [3; 2; 1]

One logical property of List.rev is:

• Calling List.rev twice must return the elements in the original order.

Here's an example assertion:

List.rev (List.rev [1; 2; 3]) = [1; 2; 3];;

val it : bool = true

A generic assertion

In the previous example List.rev was tested against an example value [1; 2; 3]. To make the assertion generic, the example value can be parameterized as any list:

fun xs -> List.rev (List.rev xs) = xs;;

val it : xs:'a list -> bool when 'a : equality = <fun:clo@33>

Hedgehog will then attempt to generate a test case that falsifies the assertion. In order to do that, it needs to know which generator to use, to feed xs with random values.

A generator for lists of integers

Values for xs need to be generated by a generator, as shown in the Generators sections. The following one is for lists of type integer:

let g = Range.constant 0 100 |> Gen.int32 |> Gen.list (Range.linear 0 20);;
val g : Gen<int list>

Every possible value generated by the g generator must now be supplied to the assertion, as shown below:

A first property

fun xs ->
    List.rev (List.rev xs) = xs
    |> Property.ofBool
|> Property.forAll g;;

val it : Property<unit>

But what is forAll? This comes from predicate logic and essentially means that the assertion holds for all possible values generated by g.

Properties can also be created using the property expression

Here's how the previous property can be rewritten:

property {
    let! xs = g
    return List.rev (List.rev xs) = xs
}

Try out (see it pass)

let g = Gen.list (Range.linear 0 100) Gen.alpha
let propConfig =
    PropertyConfig.defaultConfig
    |> PropertyConfig.withTests 500<tests>

property {
    let! xs = g
    return List.rev (List.rev xs) = xs
}
|> Property.renderWith propConfig
|> printfn "%s";;

>
+++ OK, passed 500 tests.

The above property was exercised 500 times. The default is 100, which is what Property.render does:

let g = Gen.list (Range.linear 0 100) Gen.alpha

property {
    let! xs = g
    return List.rev (List.rev xs) = xs
}
|> Property.print

>
+++ OK, passed 100 tests.

Outside of F# Interactive, you might want to use Property.check or Property.checkWith, specially if you're using Unquote with xUnit, NUnit, MSTest, or similar.

Try out (see it fail)

let tryAdd a b =
    if a > 100 then None // Nasty bug.
    else Some (a + b)

property { let! a = Range.constantBounded () |> Gen.int32
           let! b = Range.constantBounded () |> Gen.int32
           return tryAdd a b = Some (a + b) }
|> Property.render
|> printfn "%s";;

>
*** Failed! Falsifiable (after 3 tests and 24 shrinks):
101
0

The test now fails. — Notice how Hedgehog reports back the minimal counter-example. This process is called shrinking.

Reporting

Sometimes, it's useful to run a property and deal with the result programmatically. For example, if you wanted to implement custom reports, or reports for another tool to consume (JUnit reports, or an HTML representation). This can be done with the series of Property.report* functions. There are currently 8 variations of these functions:

• report — Checks a Property<unit> and creates a report.
• reportWith — Checks a Property<unit> with a specific configuration and creates a report.
• reportBool — Checks a Property<bool> and creates a report.
• reportBoolWith — Checks a Property<bool> with a specific configuration and creates a report.
• reportRecheck — Rechecks a Property<unit> and creates a report.
• reportRecheckWith — Rechecks a Property<unit> with a specific configuration and creates a report.
• reportRecheckBool — Rechecks a Property<bool> and creates a report.
• reportRecheckBoolWith — Rechecks a Property<bool> with a specific configuration and creates a report.

Internally, Hedgehog calls these functions from the Property.check* and Property.recheck* functions and pretty prints the report to stdout.

The report itself can be inspected to find metrics about the run, including the number of tests, discards and the overall result (OK, GaveUp, Failed). Upon a Failed status, more information can be retrieved about the nature of the failure. This data includes the Size, and Seed of the shrinked counter example, as well as the number of shrinks and a Journal containing log messages.

Here is an example of using Property.report to implement custom reports:

let integerGen = Gen.int32 (Range.constantBounded ())
let prop = property {
    let! x = integerGen
    return x < 100
}

let report = Property.report prop

match report.Status with
| OK -> printfn "Test succeeded! %d tests, %d discards" report.Tests report.Discards
| _ -> eprintfn "Test failed."