paper-qa
paper-qa copied to clipboard
Broken title search ut
Summary of Problem The expected vs asserted citation count was not matching, causing this UT to specifically fail.
I noticed if I switched the order of sources from semantic_scholar, crossref to crossref, semantic_scholar the citation count would be even more dramatically different (similar to the results James was noticing earlier)
More importantly, I found the papers on semantic scholar (25); which doesn't match our expected value (23); so that failure makes sense. The following assertion is failing for the same reason: 196 assertion vs 191 expected.
Vibe checked the results of this change via a CI run: https://github.com/Future-House/paper-qa/actions/runs/10859898582/job/30139826477
This does not fix the issue that James found earlier this evening in test_agents.py.
The minor variations, i.e. 196 vs 191 is expected, because the papers will get more citations over time. The bigger question is why the cassettes aren’t being used because this should come back with a recorded request each time, I’m in favor of root causing that before we merge this. Maybe we regenerated these cassettes but didn’t update the test?