Broken forms – values filled in from former scenarios

[tpraxl]

I have been researching this for some hours now and I'm stuck. I'm working with laravel 5.3. Other specs are listed below.

Calling $this->visit(route('customer.add')) returns cached page content, which is very bad when you try to test UI behavior for missing mandatory fields.

This is true for scenarios as well as outlines (example tables for scenarios). The second scenario dumps page content for filled in fields of the former scenarios although nothing has been saved.

To be clear about that behavior: With behat: I visit the form. I fill in the field, hit submit, that produces an error. The model is not saved. An error message is shown. I switch the page. Then I visit the form again. I see the last state of the form with the filled in field instead of an empty form. This includes the error message.

This behavior differs from my manual tests, where I get an empty form as expected.

I would expect $this->visit(route('myroute')) to return an empty form (which is only true for the very first scenario). But it seems to take whatever it had before.

Also tried to force reloading with $this->visit('/customer/edit?ts='.random_bytes(20)) – without success. I've added the following line to my template {{ isset($_GET['ts'])?$_GET['ts']:'x' }} just to make sure, there's no other error-source, but the output is always x when using behat (so the tsparameter is not passed). However, manual tests of course show the passed parameter value instead.

In my template, I'm using the Form facade (Collective\Html\FormFacade) like so:

{!! Form::model($customer, ['route' => 'customer.store']) !!}
{!! Form::label('first_name', 'First name:') !!}
 {!! Form::text('first_name') !!}
 {!! Form::label('last_name', 'Last name:') !!}
 {!! Form::text('last_name') !!}
{!! Form::close() !!}

Example: I have two scenarios.

The first one:

• visits page1

Result:

<label for="first_name">First name:</label>
<input name="first_name" type="text" id="first_name">

Then the first one:

• fills in first_name with "Thomas"
• hits submit (effectively shows an error message and doesn't save the model)
• changes to page2

The second scenario:

• visits page1

Result:

<label for="first_name">First name:</label>
<input name="first_name" type="text" value="Thomas" id="first_name">

BTW: This also is true when dealing with one scenario only:

• Visit page1
• Fill in a field
• Submit with error
• Visit page2
• Visit page1 -> Field is filled in. But it shouldn't be.

Also tried all of the options below, in all combinations:

/**
     * @AfterScenario
     */
    public function clearCache()
    {
        // clear compiled templates, otherwise you get forms with filled fields from former scenarios.
        //Artisan::call('view:clear');
        //Artisan::call('cache:clear');
        Cache::flush();
    }

BTW: I've also seen this pull request: https://github.com/laracasts/Behat-Laravel-Extension/pull/6

I'm stuck and I would very much appreciate any help from the great laravel community. Thanks in advance.

My routes:

Route::get('/customer/edit', 'CustomerController@add')->name('customer.add');
Route::put('/customer', 'CustomerController@store')->name('customer.store');

My .env.behat contains

CACHE_DRIVER=array
SESSION_DRIVER=array

My behat.yml

default:
    extensions:
        Laracasts\Behat:
            env_path: .env.behat
        Behat\MinkExtension:
            default_session: laravel
            laravel: ~
    suites:
        domain:
          contexts:
            - DomainContext
        ui:
          contexts:
            - UIContext

My composer.json

"behat/behat": "^3.1",
"behat/mink": "^1.7",
"behat/mink-extension": "^2.2",
"laracasts/behat-laravel-extension": "^1.0"

[tpraxl]

Well.. sorry for the complicated description above. I decided to provide a simple project that shows the problem and proves at least one of the errors.

You can find the project here: https://github.com/tpraxl/laracasts-behat-bug-example

Setup and confirmation of the bug is easy:

Setup

• Clone the project.
• Run cp .env.example .env (copies example dotenv to .env)
• Run php artisan key:generate (generates your application key in .env)

Test

• Run phpunit. The tests should succeed. This proves that laravel works as you would expect it
• Run vendor/bin/behat. The tests should fail. This proves that laracasts/behat-laravel-extension caches filled in form fields even if you change the pages in between.

If you wish to test the form behavior manually, just run php artisan serve and visit localhost:8000/plain-form in your browser.

I would very much appreciate any help here.

Thanks in advance.

[karlomikus]

Testing on my Homestead environment everything works fine:

Feature: Show cached form field values to audience
  In order to test my application
  As a developer
  I need to be sure that previously filled in form fields are not filled in when revisiting a form

  Scenario: Proof that completely filled in form is submittable # features/form-fields-behavior.feature:6
    Given I am on "/plain-form"                                 # FeatureContext::visit()
    When I fill in the following:                               # FeatureContext::fillFields()
      | first_name | Jeffrey               |
      | last_name  | Way                   |
      | email      | [email protected] |
    And I press "Submit"                                        # FeatureContext::pressButton()
    Then I should be on "/"                                     # FeatureContext::assertPageAddress()
    And I should see "Laravel"                                  # FeatureContext::assertPageContainsText()

  Scenario: Proof that incompletely filled in form is not submittable # features/form-fields-behavior.feature:16
    Given I am on "/plain-form"                                       # FeatureContext::visit()
    When I fill in the following:                                     # FeatureContext::fillFields()
      | first_name | Jeffrey |
      | last_name  | Way     |
    And I press "Submit"                                              # FeatureContext::pressButton()
    Then I should be on "/plain-form"                                 # FeatureContext::assertPageAddress()
    And I should see "The email field is required."                   # FeatureContext::assertPageContainsText()

  Scenario: Form fields, filled in previous scenarios should not be filled in in the next scenario # features/form-fields-behavior.feature:25
    Given I am on "/plain-form"                                                                    # FeatureContext::visit()
    When I fill in the following:                                                                  # FeatureContext::fillFields()
      | email | [email protected] |
    And I press "Submit"                                                                           # FeatureContext::pressButton()
    Then I should be on "/plain-form"                                                              # FeatureContext::assertPageAddress()
    And I should see "The first name field is required."                                           # FeatureContext::assertPageContainsText()
    And I should see "The last name field is required."                                            # FeatureContext::assertPageContainsText()

  Scenario: Previously filled in form fields should not be filled in when revisiting the page # features/form-fields-behavior.feature:34
    Given I am on "/plain-form"                                                               # FeatureContext::visit()
    When I fill in the following:                                                             # FeatureContext::fillFields()
      | first_name | Jeffrey |
      | last_name  | Way     |
    And I press "Submit"                                                                      # FeatureContext::pressButton()
    And I am on "/"                                                                           # FeatureContext::visit()
    And I should see "Laravel"                                                                # FeatureContext::assertPageContainsText()
    And I am on "/plain-form"                                                                 # FeatureContext::visit()
    And I fill in "email" with ""                                                             # FeatureContext::fillField()
    And I press "Submit"                                                                      # FeatureContext::pressButton()
    And I should be on "/plain-form"                                                          # FeatureContext::assertPageAddress()
    Then I should see "The email field is required."                                          # FeatureContext::assertPageContainsText()

4 scenarios (4 passed)
26 steps (26 passed)
0m1.08s (27.01Mb)

Note that I changed last scenario step to And I fill in "email" with "" otherwise the step Then I should see "The email field is required." wouldn't pass.

[tpraxl]

Thanks for looking into this!

However, I'm sorry, this is a misleading error in my test. It shouldn't say "The email field is required" at the end. It should say "The first name field is required". The email field is supposed to be filled in. Your change to "" was not intended.

Also: my test fails one step before yours. It fails on "And I should be on "/plain-form", because this is exactly what the test is supposed to show: When you fill in "first name" and "last name", submit, change the page and go to the form again, then enter the "email" and submit, you'd expect an error message. But the fields "first name" and "last name" are mistakenly still filled in and that is the bug.

I'll correct the unfortunate last step in my example project and tell you when this is done. In the meantime, please try the following. It should fail, which proves that the bug is present.

Scenario: Previously filled in form fields should not be filled in when revisiting the page
        Given I am on "/plain-form"
        When I fill in the following:
            | first_name | Jeffrey               |
            | last_name  | Way                   |
        And I press "Submit"
        And I am on "/"
        And I should see "Laravel"
        And I am on "/plain-form"
        And I fill in "email" with "[email protected]"
        And I press "Submit"
        And I should be on "/plain-form"
        Then I should see "The first name field is required."
        And I should see "The last name field is required."

[tpraxl]

@karlomikus I updated the project on https://github.com/tpraxl/laracasts-behat-bug-example Note that this test is supposed to fail, in order to show the bug. Thanks for looking into it.

[karlomikus]

Yeah, I see the problem, maybe it's because of browserkit driver or something, I'll look into it more, but for now you can use:

/**
 * @When I restart session
 */
public function iRestartSession()
{
    session()->clear();
    $this->getSession()->reload();
}

to restart the session completly.

[tpraxl]

Thanks for your hint. Great! I'll try that asap.

BTW: Originally, I had this problem even between scenarios (one scenario fills in, another scenario has to deal with filled in forms), however, the example project didn't reproduce this error, so it must have another cause.

Thanks @karlomikus !

[tpraxl]

Sorry for the delay. I tried your approach and it works. Changed the failing scenario to

Scenario: Previously filled in form fields should not be filled in when revisiting the page
        Given I am on "/plain-form"
        When I fill in the following:
            | first_name | Jeffrey               |
            | last_name  | Way                   |
        And I press "Submit"
        And I am on "/"
        And I should see "Laravel"
        And I am on "/plain-form"
        And I reset the session
        And I fill in "email" with "[email protected]"
        And I press "Submit"
        And I should be on "/plain-form"
        Then I should see "The first name field is required."
        And I should see "The last name field is required."

and implemented

/**
     * @Given /^I reset the session$/
     */
    public function iResetTheSession()
    {
        session()->clear();
        $this->getSession()->reload();
    }

Thanks for the hint. It isn't really a solution of this issue, but at least a workaround for the time being.

[alnutile]

@tpraxl in the mean time I merge in that PR from before that might help with the Cache::flush after the scenario