human-essentials icon indicating copy to clipboard operation
human-essentials copied to clipboard

[BUG]: Flaky test in children_served_report_service_spec.rb

Open elasticspoon opened this issue 10 months ago • 2 comments

See https://github.com/rubyforgood/human-essentials/pull/4273#discussion_r1564291195 for full discussion.

rspec ./spec/services/reports/children_served_report_service_spec.rb[1:2:1] --seed 56095 fails on main. rspec ./spec/services/reports/children_served_report_service_spec.rb --seed 11104 passes.

The test is flaky, it is likely related to the items that are getting created when new kits are created by factory bot in the test. Most likely some of the BaseItem are getting counted as disposables since they sometimes have "Diaper" in their names. We can also tell that the seeded items are the issue since destroying all the items at the start of the test fixes the issue.

To fix this you will need to change

      toddler_disposable_kit = create(:kit, :with_item, organization: organization)
      infant_disposable_kit = create(:kit, :with_item, organization: organization)
      infant_cloth_kit = create(:kit, :with_item, organization: organization)

to ensure that those items do not have diaper in their names (i think :slightly_smiling_face: ).

elasticspoon avatar Apr 21 '24 21:04 elasticspoon

I was pondering this, and I'm wondering if what we should do is have the generic Item base off of a 'neutral' base item -- one that doesn't fit in to any of the categories for now. Then it's not going to get counted in on any of the scopes, or in with the diapers or period supplies or adult incontinence for the kits. or the generic test kit should have no items -- but I suspect we disallow empty kits on validation.

cielf avatar Apr 22 '24 22:04 cielf

I'd rather fix this with our bigger push to stop relying on seeded / randomized data for our tests. In this case, if we care what the type of item is, we need to specify it when we create it.

dorner avatar Apr 25 '24 14:04 dorner