Add support for testing

[filippsen]

Summary

Add support for providing tests to be run against currently selected project.

Motivation

I would like to test my smart contracts by writing JavaScript unit tests

Additional context

• Enable developers to improve coding skills (reviewing, reassessing, thinking from different perspective)
• Enable developers to check for assumptions before deployment
• Enable developers to learn by testing (experimentation)

Tasks

Test Runner

To enable inspecting all the tests written for the currently open project.

• [x] New tests directory

  • [x] filesystem functionality - ~~In progress in refactoring branch and https://github.com/SuperblocksHQ/superblocks-lab/issues/166~~
    • [x] located in Files hierarchy (left side menu)
    • [x] Ability to create a new file
    • [x] Ability to remove an existing file
    • [x] Ability to rename an existing file
    • [x] Ability to edit file contents
    • [x] Merge latest master (version 1.1.0) into feature/testing-GUI. New branch created: feature/master-testing

• [x] New Tests button

  • [x] GUI
    • [x] presented at the bottom of the GUI
    • [x] Opens Tests panel

• [x] New Tests panel

  • [x] GUI

    • [x] A panel, not a window

    • [x] Contains "X" close button

    • [x] Contains a "control bar"

      • [x] GUI
        • [x] Run all tests button (green play icon)
        • [x] Toggle file watching button (yellow lightning icon)
        • [x] Stop all tests button (red stop icon)
      • [x] define expected test file format (example to serve as reference) - @filippsen
      • [x] ability to run arbitrary test file(s) taking an example as reference input - @filippsen
        • [x] Requires fine-grained control over which tests are selected for running (preparation)
        • [x] Requires access to compiled data or compiler, including constructor arguments
        • [x] Demands handling contract deployment
        • [x] Customized network settings
        • [x] Customized account settings
        • [x] Requires access to deployed data or deployer
        • [x] A custom reporter (non HTML) would be ideal
        • [x] Interact with the compiler (to compile arbitrary test files)
        • [x] Replace reference testing account data with Account settings
        • [x] Access the new filesystem (in order to read arbitrary test files from it)
        • [x] Replace reference testing contracts data with contract context (dappfile)
      • [x] ability to run a specific test block described in any arbitrary test file at any time - @filippsen
        • Fine grained control over when tests get called (from previous step - preparation)
      • [ ] ability to cancel any currently running tests
      • [x] ability to automatically re-run on change (watch functionality)

    • [x] Contains the tests list

      • [x] GUI
        • [x] Summarizes the status of last run
        • [x] Scrollable and collapsable list of all available tests files
        • [x] Scrollable and collapsable list of all test blocks described in each test file
        • [x] Ability to select a specific test
        • [x] Informs about success or failure of each test (icon)
        • [x] Informs about success or failure of each test (colored selection rectangle)
        • [x] Informs about time taken to complete each test
      • [x] ability to extract a detailed list of all tests described in a given test file
      • [x] ability to extract detailed test output data for each test in any given test file (test results)
        • To be used as input for custom rendering and styling

    • [x] Contains a "Test Summary" bar

      • [x] GUI
        • [x] Lists aggregated test statistics
          • [x] Informs about total test count
          • [x] Informs about total number of passed tests
          • [x] Informs about total number of failed tests
      • [x] ability to maintain statistics considering all test output data

    • [x] Contains a "Console Output"

      • [x] GUI
        • [x] Outputs selected test details
          • [x] progress details
          • [x] success
          • [x] test failure or other error
          • [x] cancelled/stopped
      • [x] ability to extract detailed test output data for each test in any given test file (test results)

• [x] Complete interaction with any test file from /tests/

• [x] Solidify low-level test run calls (Security ? Correctness ? Speed ?)

  • [x] Move test execution to sandboxed iframe
  • [x] Enable web3 object inside sandboxed iframe

• [ ] Fix all that is marked as TODO

• [ ] Resolve all FIXME occurrences

• [x] Properly format the new code added during the early stages (legacy)

[filippsen]

Initial test reference (vanilla version)

This reference aims to: a. Demonstrate what a working test example using Web3 would look like; b. Analyze what would be the required dependencies for running it; c. Analyze what would be the inputs necessary to enable testing arbitrary contracts.

This leads to extensive and verbose code (read: explicit and raw). The testing code could be further compressed into a simpler API, focused on providing shortcuts to the most commonly used operations.

The example test reference is based on the HelloWorld.sol contract from Hello World project template.

Dependencies

• ABI
• Mocha
• Tx
• Web3

Inputs

• contract ABI (or contract source code and access to the compiler)
• contract binary (or contract source code and access to the compiler)
• valid account address (or access to existing wallet)
• corresponding account private key (or access to existing wallet)

Example

Note: some details were suppressed for readability.

describe('User test action: manually check contract data', function (done) {
    var contractInstance;

    beforeEach(function (done) {
        const contractBin = "0x6060...";
        const contractABI = instance.abi;

        web3.eth.getTransactionCount(account_address, function(error, result) {
            [...]
            const tx = new Tx[...]
            web3.eth.sendRawTransaction("0x"+tx.serialize().toString("hex"),
                [...]
                getTransactionReceipt(currentContractTransactionHash, function(err, res) {
                    [...]
                    var contract = web3.eth.contract(contractABI);
                    contractInstance = contract.at(res.contractAddress);
                    done();
                });
            });
        });
    });

    it('matches message data', function (done) {
        var expectedValue = "Hello World!";
        contractInstance.message(function(error, result) {
            if(error) {
                console.error(error);
                done(error);
            } else {
                if(result !== expectedValue) {
                    done(new Error(result));
                } else {
                    done();
                }
            }
        });
    });

    it('update message data', function (done) {
        [...]
        web3.eth.getTransactionCount(account_address, function(error, result) {
            if(error) {
                done(new Error("Could not get nonce for address " + account_address));
            } else {
                account_nonce=result;
                var data = ABI.ABI.simpleEncode("update(string)", "Super Hello World!");
                [...]
                const tx = new Tx[...]
                web3.eth.sendRawTransaction("0x"+tx.serialize().toString("hex"),
                    function(error, result) {
                        if(error) {
                            console.error(error);
                            done(error);
                        } else {
                            contractInstance.message(function(error, result) {
                                var expectedValue = "Super Hello World!";
                                if(error) {
                                    console.error(error);
                                    done(error);
                                } else {
                                    if(result !== expectedValue) {
                                        done(new Error(result));
                                    } else {
                                        [...]
                                        done();
                                    }
                                }
                            });
                        }
                    }
                );
            }
        });
    });
});

[filippsen]

Feedback 592e5813 ("dynamic result data is now rendering")

Main objective

Get a working demonstration against the hardcoded reference contract and tests with all the UI elements in presentable state, according to the original design.

Pending issues

• [x] F01. It looks like the "selection" functionality is expected to be a single choice (not multiple choices). The reason for that is because it serves for inspecting the output and repeating a single test, for instance. (Original item: "Ability to select a specific test").
• [x] F02. The list of tests begins pre-filled with data. It should be empty at the start. Also, consider a reset mechanism in code. @filippsen
• [x] F03. The contract being tested and the list of tests are not being built based on the test results. It should be read from the test output instead i.e. there is no FundRaise in the included example. (Original items: "Scrollable and collapsable list of all available tests files", "Scrollable and collapsable list of all test blocks described in each test file" and "Informs about time taken to complete each test").
• [ ] F04. It's likely to be expected that the action buttons are always visible and accessible. Consider detaching the buttons from the list of tests (they already are separated into two "columns", after all). The way it is now they scroll together with the set of items.
  More information: the Tests panel is divided into two main elements: left and right. The left side should be further split up into 3 sections: left column with buttons; top Done counter; main list of tests. Example:

<vertical=false split>
    <div with TestControls>
    <div with vertical=true>
      <div with Done counter>
      <div with list of tests>
...

• [x] F05. When moving the cursor over the "down arrow", for expanding/collapsing the items, the cursor doesn't change (inconsistency across different clickable actions. Please observe the left side menu - Files hierarchy for reference).

• [x] F06. Console output reads "NOTE - Console output from this specific test" on success. Consider a clear success message instead (to be defined later). (Original item: "Outputs selected test details") @filippsen

• [ ] F07. Listed items text overlap with the time taken information when resizing the test list (to the left, decreasing the width) More information: It is possible to set the text-overflow attribute to testResults in src/components/projecteditor/style.less. However, the dynamic width must be considered.

• [x] F08. Consider setting a minimum width to the list of items. As it is now, it is possible to lose access to the resize action (gets hidden behind another element when resizing all the way to the left). Please observe the Files hierarchy and take it as a reference guideline (for consistency).

• [x] F09. Consider a maximum width for the list of items container.

• [x] F10. It is possible to scroll down the Tests panel, hiding the black "title" bar. Consider setting the black bar to always be visible for clarity (title makes the context clear) and accessibility (the "X" close button is visible). It is unclear why there is a scroll bar there to begin. Previous feedback from Javier reads: "There is a scrollbar for the entire pane. That should not be the case as the pane itself should not be scrollable but only the container within".

• [x] F11. The current test "selection" (choice) should be passed on as argument to the runSingle call. That would repeat the last set of tests only running the current selection. There is an in-code comment hinting for that. @filippsen

• [x] F12. The "selection" color depends on the test state i.e. green on success (not red). (Original item: " Informs about success or failure of each test - colored selection rectangle") @filippsen

• [x] F13. Test Summary text should be set bold (or a different font, which is hard to tell without seeing the design details. Please refer to the original design material).

• [ ] F14. When toggling the Tests panel the editor still flickers. Experiment toggling the Tests panel while other windows are open. There is apparently a "redraw all" action happening.

More information: in panes.js, the div with key set topanes2 changes position, depending whether the Tests panel is open or not. There is an in-code comment about that, which helps confirming the issue: {/*remove this condition and add proper state management for handling closing the pane.*/}

• [ ] F15. Reiterating the situation that happens when multiple windows open behind the Tests panel: with the Tests panel open, clicking the Deploy button results in a weird situation (multiple scroll bars. Easy to lose track where things are or went to). This behavior differs from Transactions History, which adapts to everything.

• [x] F16. When resizing the browser window, the list of tests container apparently does not redraw correctly (or at all ?). This results in artifacts on the final "image" (dark colored / blurred portion of the screen. Only in Tests panel). Reproducible by opening up the browser console output, moving it to the bottom of the screen and resizing it up and down. Update: unable to reproduce in both Chrome or Firefox.

• [x] F17. Reminder: please consider more appropriate naming for data. References to person and persons are still around.

• [x] F18. Selecting the text on the list of tests panel and dragging the selection to the right side, scrolls the contents horizontally.

• [x] F19. Unstyled panel scrollbar in Firefox More information: it matches other scroll bars. Filed a separate issue: https://github.com/SuperblocksHQ/superblocks-lab/issues/265

• [ ] F20. Unaligned milliseconds text when resizing list of tests More information: the test item element does not cover the maximum possible width.

• [x] F21. Unimplemented grand total time spent

[filippsen]

Issues master-testing ("merge to post-refactoring code base")

• [x] M01. Test item list is not working
• [x] M02. Unsupported Tests button (new BottomBar code)

[filippsen]

New execution environment ("safe" code path parity with non-safe environment)

• [x] S01. Enable proper state reset when rerunning (Hypothesis: only works as expected on the first try, on fresh start ?). Solved with commit 2b34d072
• [x] S02. To solve: running multiple tests are not straightforward in the new isolated environment. To consider: keep a single iframe and combine results before rendering or create a new isolated execution unit for every new test file run ? (Hypothesis: subsequent tests are replacing existing ones)
• [x] S03. Further consolidate initialization state to prevent mocha from ever fumbling during asynchronous execution (related: S01) Expected to be working better after the new safeRun codepath.
• [x] S04. Consider case where mocha might fail to completely run after regular reload
• [x] S05. Enable back single test run now in safe environment
• [x] S06. Moved to P04
• [x] S07. Consolidate endpoints ?
• [x] S08. Add support to accounts (make sure it is still cooperating with existing wallet) It is using the same hardcoded account and balance changes are reflected on the wallet.
• [x] S09. Investigate why "No tests available" only appear after project load (or when switching)

---

• [x] B01. There might be a chance of test runner not having enough time to setup before starting next execution after short-lived or no-op test operations Unable to replicate
• [x] B02. There might be a chance of test errors impacting the final output data (Investigate possible interleaving ?) Update: not in standard testing. It might have been a problem with older codepath in demo mode.
• [ ] B03. Accounts are expected to reset on re-run (as with everything else), but the nonce is currently left untouched. Update: web3 accounts get out of sync when rerunning.
• [x] B04. Moved to P05

---

• [ ] P01. Synchronize with latest master (merge)
• [ ] P04. Remove saferun code path and make it safe-only by default
• [ ] P02. Use the new Upload functionality to create a reference project to be used for validation
• [ ] P03. Consider segregating error output and showing the appropriate results upon selection. As it is, only a single output is taken into account.
• [ ] P05. Better handle the possibility of writing cpu intensive user code that blocks the entire application

[filippsen]

Unassigning as requested.