Define one page object for each state in your UI. A state can be anything from a page, or an individual modal, or the same page as a different state, but altered, for example a logged in user.

The possible custom operations are:

• name: Name of the state. Required
• localState: A type to store some local state. Will be recreated everytime this page object is entered
• onEnter: Function to run when entering this page. Only one allowed
• onExit: Function to run when exiting this page. Only one allowed
• transition: Possible transition. Define how to transition to the next state, as well as which state to navigate to. Any number of transitions allowed
• action: Possible action. Define function to run while in this page state. Any number of actions allowed
• exitAction: After scrutiny traverses the graph of possible states, will pick a single exit action defined in any state to navigate to and finish the test with

A page looks like this:

let loggedInComment = fun (globalState: GlobalState) ->
    page {
        name "Logged In Comment"

        localState (LoggedInComment())

        onEnter (fun ls ->
            printfn "Checking on page comment"
            // Do something with LocalState e.g. set the HomeLink property
            ls.HomeLink <- "#home"
            "#header" == "Comments"
        )

        onExit (fun _ ->
            printfn "Exiting comment"
        )

        transition ((fun ls -> click ls.HomeLink) ==> home)
        transition ((fun _ -> click "#signin") ==> signIn)

        action (fun _ -> () /*do something on the page*/)
        action (fun _ -> () /*do something else on the page*/)

        exitAction (fun _ -> () /*final action to perform before exiting the test*/)
    }

The name must be unique. Any number of transitions and any number of actions can be defined. The exitAction is optional, and multiple pages can have an exitAction. If multiple are defined, Scrutiny will randomly choose one to perform.

The GlobalState in the example is any type defined in your test that you can use to pass data between states, e.g. Username or IsLoggedIn

The LocalState is specific to a state, and is constructed each time that state is visited. It's optional, but when set, all functions will have access to the local state via the function parameter. In the above example, localState is defined as:

type LoggedInComment() =
    member val Comment = String.Empty with get, set

Configuration

Some things can be configured via ScrutinyConfig. The default config is:

{ ScrutinyConfig.Seed = Environment.TickCount
  MapOnly = false
  ComprehensiveActions = true
  ComprehensiveStates = true
  ScrutinyResultFilePath = Directory.GetCurrentDirectory() + "/ScrutinyResult.html"
  Logger = printfn "%s" }

Seed is printed during each test to be able to recreate a specific test run. MapOnly won't run the test at all, but only generate the HTML Graph report. ComprehensiveActions will run ALL defined actions anytime it enters a state with actions defined. If false, it will run a random subset of actions. ComprehensiveStates will visit ALL states in the state machine. If this is false, then it will visit at least half of all states before randomly quitting. ScrutinyResultFilePath is the directory and specified file name that the generated HTML report will be saved in Logger is how individual messages from scrutiny will be logged. The signature is string -> unit. This is useful for things like XUnit that bring their own console logging mechanism, or if you wanted to integrate a larger logging framework.

To actually run the test, call the scrutinize function with your entry state, config, and global state object. e.g.

// Sample Global State. This can be anything, and all page states will receive the same instance
type GlobalState() =
    member val IsSignedIn = false with get, set
    member val Username = "MyUsername" with get, set
    member val Number = 42

[<EntryPoint>]
let main argv =
    let options = FirefoxOptions()
    do options.AddAdditionalCapability("acceptInsecureCerts", true, true)

    use ff = new FirefoxDriver(options)
    let currentDirectory = DirectoryInfo(Directory.GetCurrentDirectory())

    let config =
        { ScrutinyConfig.Default with
              Seed = 553931187
              MapOnly = false
              ComprehensiveActions = true
              ComprehensiveStates = true
              ScrutinyResultFilePath = currentDirectory.Parent.Parent.Parent.FullName + "/myResult.html" }

    // Start tests. In this case we're using CanopyUI, but can be any test runner e.g. XUnit or Expecto
    // Start CanopyUI tests
    "Scrutiny" &&& fun _ ->
        printfn "opening url"
        url "https://localhost:5001/home"

        let gs = GlobalState()

        // The call to start Scrutiny, and construct a graph and "click" through all states
        scrutinize config gs home
        // or
        // scrutinizeWithDefaultConfig gs home

    switchTo ff
    pin canopy.types.direction.Right

    run()
    quit ff

    0

At the end of the run, Scrutiny will return an object which contains the generated adjacency graph, as well as a list of individual steps taken, along with the actions performed in each state.

Important note for F# users

As the transitions ultimately depict a cyclic graph, it is necessary to declare module or namespace as recursive so that pages defined later can be referenced by pages earlier. Note the usage of the rec keyword. e.g.:

module rec MyPages =
    let firstPage = fun (globalState: GlobalState) ->
        page {
            name "First Page"
            transition ((fun () -> click "#second") ==> secondPage)
        }

    let secondPage = fun (globalState: GlobalState) ->
        page {
            name "Second Page"
            transition ((fun () -> click "#first") ==> firstPage)
        }

Define one class for each state in your UI, and decorate it with the PageState attribute. A state can be anything from a page, or an individual modal, or the same page as a different state, but altered, for example a logged in user.

The possible attributes are:

• PageState: Define a class as a Page state.
• OnEnter: Function to run when entering this page. Only one allowed
• OnExit: Function to run when exiting this page. Only one allowed
• TransitionTo: Possible transition. Define how to transition to the next state, as well as which state to navigate to. Any number of transitions allowed
• Action: Possible action. Define function to run while in this page state. Any number of actions allowed
• ExitAction: After scrutiny traverses the graph of possible states, will pick a single exit action defined in any state to navigate to and finish the test with

A PageState could look like this:

using Scrutiny.CSharp;

[PageState]
public class LoggedInComment
{
    private readonly GlobalState globalState;
    private string localComment = string.Empty;

    public LoggedInComment(GlobalState globalState)
    {
        // Construct anything necessary.
        // The constructor is called everytime Scrutiny navigates to this state
    }

    [OnEnter]
    public void OnEnter()
    {
        // Do something when scrutiny enters this state
        // Can optionally be async/await
        // Can only define one
    }

    [Action]
    public async Task WriteComments()
    {
        // Do something on the page
        // Can optionally be non-async
        // Define any number of these
    }

    [OnExit]
    public void OnExit()
    {
        // Do something when scrutiny exits this state
        // Can optionally be async/await
        // Can only define one
    }

    [ExitAction]
    public async Task ExitAction()
    {
        // One exit actions amongst all page states is chosen
        // Define any number of these
        // Can optionally be non-async
    }

    [TransitionTo(nameof(AnotherState))]
    public void TransitionToAnotherState()
    {
        // Code to perform state transition
        // Define any number of these
        // Can optionally be async/await
    }
}

Available attriutes are: PageState This decorates the class. Scrutiny will search for all PageStates within an assembly. OnEnter Only one allowed per class. This method will be run anytime Scrutiny enters this state. OnExit Only one allowed per class. This method will be run anytime Scrutiny exits this state. ExitAction Once Scrutiny is done navigating through the states and actions, it will randomly choose a single exit action out of all defined exit actions to exit the state machine. Action These are any actions that are performed within a state, and then stay in the same state. TransitionTo(string) These are any methods that perform state transitions. Pass the name of another PageState to tell Scrutiny where the transition goes to.

Configuration

Some things can be configured via the Scrutiny.CSharp.Configuration.Configuration POCO. The default config is:

Seed = Environment.TickCount
MapOnly = false
ComprehensiveActions = true
ComprehensiveStates = true
ScrutinyResultFilePath = Directory.GetCurrentDirectory() + "/ScrutinyResult.html"
Logger = (Action<string>)((s) => Console.WriteLine(s))

Seed is printed during each test to be able to recreate a specific test run. MapOnly won't run the test at all, but only generate the HTML Graph report. ComprehensiveActions will run ALL defined actions anytime it enters a state with actions defined. If false, it will run a random subset of actions. ComprehensiveStates will visit ALL states in the state machine. If this is false, then it will visit at least half of all states before randomly quitting. ScrutinyResultFilePath is the directory and specified file name that the generated HTML report will be saved in Logger is how individual messages from scrutiny will be logged. This is useful for things like XUnit that bring their own console logging mechanism, or if you wanted to integrate a larger logging framework.

To actually run the test, call the Scrutiny.CSharp.Scrutinize.Start<Home>(gs, config) method. It takes your entry state as a generic type argument, and a constructed global state object as well as your config as parameters.

using Scrutiny.CSharp;

[Fact]
public async Task WithAttrs()
{
    var browser = await playwright.Firefox.LaunchAsync(headless: false);
    var context = await browser.NewContextAsync(ignoreHTTPSErrors: true);
    var page = await context.NewPageAsync();

    await page.GoToAsync("https://127.0.0.1:5001/home");

    var config = new Configuration
    {
        Seed = 553931187,
        MapOnly = false,
        ComprehensiveActions = true,
        ComprehensiveStates = true
    };

    var gs = new GlobalState(page, outputHelper);
    var result = Scrutinize.Start<Home>(gs, config);

    Assert.Equal(7, result.Steps.Count());
    Assert.Equal(5, result.Graph.Count());
}

The global state can be any class you want it to be. Scrutiny will pass the instance that is passed into the start around to each PageState it visits. At the end of the run, Scrutiny will return an object which contains the generated adjacency graph, as well as a list of individual steps taken, along with the actions performed in each state.

Ethereum: 0x05f231D19c19A2111fe03c923F26813Bad43B57f

Cardano ADA: addr1qx35nmy62dfp3n5tqgga92gxcnq5vkvflw963yg7fm5e5my68x9frc2qq0r8nstjtnjcrcnpmtpzwvp0sqz46y4ykrmqrd4dg9