inline-snapshot icon indicating copy to clipboard operation
inline-snapshot copied to clipboard

`External` base `__eq__` causes test failure in teardown after new snapshot created

Open lmmx opened this issue 6 months ago • 1 comments

I'm seeing created values (using a custom Format #311) leading to increment of state().missing_values where it should not.

  • Pytest reports 2 results for 1 test run: a pass in the test and a fail in teardown
  • Related: it also edits the files when not passed --inline-snapshot=create https://github.com/lmmx/inline-snapshot-phash/issues/2

I suspect I am missing some component to do with the proper creation of a test and decrement of the missing_values count that leads to a hard fail in teardown (some sort of sanity check that that counter was 0 and expectation that if it wasn't the test would have already bailed out and failed during the run).

I looked for where missing_values is handled and since it's never decremented, the only way to avoid it being non-zero is to never increment it.

I suspect that there might be an ordering bug here:

https://github.com/15r10nk/inline-snapshot/blob/a108d2d8e22fd60d1c95a042a1e3d9f9ffe5bbc0/src/inline_snapshot/_external/_external_base.py#L66-L71

Specifically I suspect that if you take the state create path, then you do not want to increment the missing values (it is on the line before) as this will cause your test that updated the empty external snapshot to fail.

So it would be

        if self._is_empty():
            self._assign(other)
            if state().update_flags.create:
                return True
            state().missing_values += 1  # moved after the early return in create mode
            return False

Error message nit

As a secondary note, the flag is wrong in the error message, perhaps this has not been seen in a while since early on in development

_____________________________________________________ ERROR at teardown of test_red_square_one ______________________________________________________
your snapshot is missing one value.
If you just created this value with --snapshot=create, the value is now created and you can ignore this message.

Demo

Before the change:

(inline-snapshot-phash) louis 🌟 ~/dev/inline-snapshot-phash $ python -m pytest demo/ -k one
================================================================ test session starts ================================================================
platform linux -- Python 3.14.0rc2, pytest-8.4.2, pluggy-1.6.0 -- /home/louis/dev/inline-snapshot-phash/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/louis/dev/inline-snapshot-phash
configfile: pyproject.toml
plugins: inline-snapshot-0.30.1, anyio-4.11.0
collected 3 items / 2 deselected / 1 selected                                                                                                       

demo/demo_test.py::test_red_square_one PASSED                                                                                                 [100%]
demo/demo_test.py::test_red_square_one ERROR                                                                                                  [100%]

══════════════════════════════════════════════════════════════════ inline-snapshot ══════════════════════════════════════════════════════════════════
───────────────────────────────────────────────────────────────── Create snapshots ──────────────────────────────────────────────────────────────────
╭──────────────────────────────────────────────────────────────── demo/demo_test.py ────────────────────────────────────────────────────────────────╮
│ @@ -84,7 +84,7 @@                                                                                                                                 │
│                                                                                                                                                   │
│                                                                                                                                                   │
│  def test_red_square_one(red_square: Path):                                                                                                       │
│      """Basic usage: 100px red square gets a phash."""                                                                                            │
│ -    assert red_square.read_bytes() == external("phash:")                                                                                         │
│ +    assert red_square.read_bytes() == external("phash:AAAAAAAAAAA.ph")                                                                           │
│                                                                                                                                                   │
│                                                                                                                                                   │
│  def test_red_square_two(red_square_tiny: Path):                                                                                                  │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────── phash:AAAAAAAAAAA.ph ───────────────────────────────────────────────────────────────╮
│ <binary file (334 bytes)>                                                                                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
These changes will be applied, because you used create



====================================================================== ERRORS =======================================================================
_____________________________________________________ ERROR at teardown of test_red_square_one ______________________________________________________
your snapshot is missing one value.
If you just created this value with --snapshot=create, the value is now created and you can ignore this message.
============================================================== short test summary info ==============================================================
ERROR demo/demo_test.py::test_red_square_one - Failed: your snapshot is missing one value.
===================================================== 1 passed, 2 deselected, 1 error in 0.35s ======================================================
(inline-snapshot-phash) louis 🌟 ~/dev/inline-snapshot-phash $ 

After the change:

(inline-snapshot-phash) louis 🌟 ~/dev/inline-snapshot-phash $ python -m pytest demo/ -k one
================================================================ test session starts ================================================================
platform linux -- Python 3.14.0rc2, pytest-8.4.2, pluggy-1.6.0 -- /home/louis/dev/inline-snapshot-phash/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/louis/dev/inline-snapshot-phash
configfile: pyproject.toml
plugins: inline-snapshot-0.30.1, anyio-4.11.0
collected 3 items / 2 deselected / 1 selected                                                                                                       

demo/demo_test.py::test_red_square_one PASSED                                                                                                 [100%]

══════════════════════════════════════════════════════════════════ inline-snapshot ══════════════════════════════════════════════════════════════════
───────────────────────────────────────────────────────────────── Create snapshots ──────────────────────────────────────────────────────────────────
╭──────────────────────────────────────────────────────────────── demo/demo_test.py ────────────────────────────────────────────────────────────────╮
│ @@ -84,7 +84,7 @@                                                                                                                                 │
│                                                                                                                                                   │
│                                                                                                                                                   │
│  def test_red_square_one(red_square: Path):                                                                                                       │
│      """Basic usage: 100px red square gets a phash."""                                                                                            │
│ -    assert red_square.read_bytes() == external("phash:")                                                                                         │
│ +    assert red_square.read_bytes() == external("phash:AAAAAAAAAAA.ph")                                                                           │
│                                                                                                                                                   │
│                                                                                                                                                   │
│  def test_red_square_two(red_square_tiny: Path):                                                                                                  │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────── phash:AAAAAAAAAAA.ph ───────────────────────────────────────────────────────────────╮
│ <binary file (334 bytes)>                                                                                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
These changes will be applied, because you used create



========================================================== 1 passed, 2 deselected in 0.35s ==========================================================

lmmx avatar Oct 22 '25 17:10 lmmx

I've rolled this into #313 but happy to break it out into a focused PR if preferable to review/merge separately.

  • fix: move missing_values++ to after create mode early return ed6c655

lmmx avatar Oct 22 '25 17:10 lmmx