human-essentials
human-essentials copied to clipboard
[BUG]: Flaky test in children_served_report_service_spec.rb
See https://github.com/rubyforgood/human-essentials/pull/4273#discussion_r1564291195 for full discussion.
rspec ./spec/services/reports/children_served_report_service_spec.rb[1:2:1] --seed 56095
fails on main.
rspec ./spec/services/reports/children_served_report_service_spec.rb --seed 11104
passes.
The test is flaky, it is likely related to the items that are getting created when new kits are created by factory bot in the test. Most likely some of the BaseItem
are getting counted as disposables since they sometimes have "Diaper" in their names. We can also tell that the seeded items are the issue since destroying all the items at the start of the test fixes the issue.
To fix this you will need to change
toddler_disposable_kit = create(:kit, :with_item, organization: organization)
infant_disposable_kit = create(:kit, :with_item, organization: organization)
infant_cloth_kit = create(:kit, :with_item, organization: organization)
to ensure that those items do not have diaper in their names (i think :slightly_smiling_face: ).
I was pondering this, and I'm wondering if what we should do is have the generic Item base off of a 'neutral' base item -- one that doesn't fit in to any of the categories for now. Then it's not going to get counted in on any of the scopes, or in with the diapers or period supplies or adult incontinence for the kits. or the generic test kit should have no items -- but I suspect we disallow empty kits on validation.
I'd rather fix this with our bigger push to stop relying on seeded / randomized data for our tests. In this case, if we care what the type of item is, we need to specify it when we create it.