beam icon indicating copy to clipboard operation
beam copied to clipboard

test: add playwright tests

Open Alchez opened this issue 1 year ago • 3 comments

Closes #193


Test locally inside a virtualenv using pytest ./beam/tests/mobile/test_mobile.py --browser firefox --headed --disable-warnings

Alchez avatar Nov 27 '24 11:11 Alchez

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py10100% 
beam
   barcodes.py793456%20, 22, 24, 27, 38–39, 41–42, 47–48, 50–51, 53–55, 61, 71, 79, 86, 89–90, 96–97, 103, 109, 115, 121, 128, 131–134, 136–137
   handling_unit.py814346%26, 32, 64–66, 69, 72–76, 84, 95–96, 102–103, 105–106, 108–109, 111–112, 114, 116–119, 121–123, 125, 127, 129–130, 135, 137, 141, 143–144, 147, 149–150, 158
   printing.py826421%18–19, 32–43, 45–48, 57–61, 65–66, 68–69, 78–82, 85–86, 92–94, 101–102, 104–106, 108–109, 111, 116, 118–119, 134–139, 141–150
beam/demand
   demand.py3389472%51, 66, 123, 209, 243, 290, 293, 425–426, 428–429, 432–433, 435, 439–444, 446, 448–449, 452, 457, 459–464, 466, 468, 471, 474, 479, 483–485, 490–491, 493, 495–497, 499, 505, 507–510, 512–515, 517, 519, 522–523, 525–526, 529, 543, 560–561, 563–565, 621, 625, 662, 667, 674, 695–696, 707, 713–729, 853
   receiving.py1251786%79, 131, 173, 214, 244, 249–260
   sqlite.py59198%40
   utils.py59788%56, 67, 72–75, 83
beam/doctype/beam_mobile_route
   beam_mobile_route.py30100% 
beam/doctype/beam_settings
   beam_settings.py281739%35–37, 40–41, 43, 50, 55–58, 66–68, 72–74
beam/doctype/handling_unit
   __init__.py9366%12–14
   handling_unit.py150100% 
beam/doctype/warehouse_types
   warehouse_types.py30100% 
beam/overrides
   sales_order.py10640%11–16
   stock_entry.py671873%61–62, 67–74, 80–81, 97, 112, 123, 129, 132–133
   work_order.py10280%14, 16
beam/report/demand_map
   demand_map.py35350%4–5, 7–8, 11–12, 15–16, 103–111, 127, 129–143, 146–147
beam/report/handling_unit_traceability
   handling_unit_traceability.py44440%4–6, 9–11, 30–32, 34–35, 37–38, 41–42, 45–46, 48–49, 51–52, 54–55, 57, 63–65, 67–69, 71, 74–77, 79–81, 83–86, 89–90
beam/scan
   __init__.py1432880%21, 25, 31, 39, 90, 93–94, 118–119, 129, 131–132, 136, 140–141, 144, 155–156, 160, 181, 185, 214, 216, 235–236, 242, 245, 248
   config.py28280%4, 6, 9–17, 19–21, 23–26, 28–30, 32–35, 38–39, 41
   user_login.py31310%4–6, 8, 11–13, 17–19, 21–22, 24–25, 27–29, 31–33, 35–38, 40–45, 47
www/beam
   __init__.py14140%4–7, 10–19
   index.py770%4, 6, 9–13
TOTAL127149361% 

github-actions[bot] avatar Dec 02 '24 10:12 github-actions[bot]

@agritheory I'm seeing some weird behaviour that I'm not sure if you've faced before. While running the tests locally, I'm scanning and saving the record and it creates an entry in the database. However, the pytest runner can't seem to fetch that record for assertions.

Have you seen something like this?

Alchez avatar Jan 30 '25 11:01 Alchez

@agritheory

Some things I tried:

  • Commit after SAVE
page.get_by_text("SAVE", exact=True).click() 
frappe.db.commit()
  • Time after SAVE
page.get_by_text("SAVE", exact=True).click()
time.sleep(2)
  • Scope function in db_instance()
@pytest.fixture(scope="function", autouse=True) def db_instance():
  • Re-connect after SAVE
page.get_by_text("SAVE", exact=True).click() 
currentsite = "test_site"
sites = Path(get_bench_path()) / "sites"
if (sites / "common_site_config.json").is_file():
    currentsite = json.loads((sites / "common_site_config.json").read_text()).get("default_site")

frappe.init(site=currentsite, sites_path=sites, force=True)
frappe.connect()
  • Disabled delete_draft_records temporarily to see if it is interfering.
  • Check that the URL from setup is the same that the URL used by Playwright.
  • Combinations of the above.

fproldan avatar Mar 06 '25 13:03 fproldan

I found that when the browser's transaction ends, the data appears when I use bench console.

After researching and debugging, I noticed that the test code and the browser work as two different users consulting the same database but operating in two different transaction "snapshots". When the browser makes API calls and saves data via HTTP requests, the test code needs to refresh its transaction to see the recently committed data.

frappe.db.rollback()  # End current transaction (from browser)
frappe.db.begin()     # Start new transaction with updated snapshot

Demo

https://github.com/user-attachments/assets/7d616f30-df8f-4c3f-86ae-e0a273cb1e5e

lauty95 avatar Jun 27 '25 13:06 lauty95

@Alchez Sorry if I'm not clear - this isn't my strongest area and my responses are based on some research + using print statements for debugging 😅

It's not actually "multiple user connections" - it's the same database connection but different transaction scopes.

Flow breakdown:

  1. Pytest starts > Transaction A begins here: https://github.com/agritheory/beam/blob/7ed1c82f201fa7782795cba2e63943a01cb0faa2/beam/tests/mobile/test_receive.py#L18 test_complete_partial_receipt > conftest.py > db_instance > frappe.connect() https://github.com/agritheory/beam/blob/7ed1c82f201fa7782795cba2e63943a01cb0faa2/beam/tests/conftest.py#L45

  2. Browser clicks > HTTP POST to Frappe server: https://github.com/agritheory/beam/blob/7ed1c82f201fa7782795cba2e63943a01cb0faa2/beam/tests/mobile/test_receive.py#L60

  3. Frappe server > Transaction B: BEGIN > INSERT > COMMIT > END (Server creates new transaction for the HTTP request, saves data, commits immediately)

  4. Pytest queries > Still in Transaction A - can't see Transaction B's commits: https://github.com/agritheory/beam/blob/7ed1c82f201fa7782795cba2e63943a01cb0faa2/beam/tests/mobile/test_receive.py#L64 (Returns empty because Transaction A sees old snapshot)

Solution:

frappe.db.rollback()  # End Transaction A
frappe.db.begin()     # Start a new transaction with fresh snapshot
receipts = frappe.get_all(...)  # Now sees data from Transaction B

lauty95 avatar Jul 02 '25 13:07 lauty95

I think we need a helper function here to explicitly acquire the current transaction scope. I'd personally want to use an API that is one of a context manager or a decorator to be clear I was doing this. This would require no changes in other tests.

def test_ui_thing(example_pytest_fixture):
    with use_current_db_transaction():
      value = frappe.get_value('Doctype', ...)
      assert value == True

It may have to be like this:

def test_ui_thing(example_pytest_fixture):
    with use_current_db_transaction() as _db:
      value = _db.get_value('Doctype', ...)
      assert value == True
@use_current_db_transaction()
def test_ui_thing(example_pytest_fixture):
   value = frappe.get_value('Doctype', ...)
   assert value == True

Or just a function call, which is to me, least clear.


def test_ui_thing(example_pytest_fixture):
    use_current_transaction()
    value = frappe.get_value('Doctype', ...)
   assert value == True

The concern with using a decorator is that ordering matters, it can have unintended side affects and it's the least explicit/obvious/clear when reading and reviewing code. I'm slightly in favor of the context manager API.

I'd like to prototype this code here and eventually move it to test_utils.

agritheory avatar Jul 02 '25 13:07 agritheory

@agritheory @Alchez I guess this is ready

lauty95 avatar Aug 21 '25 18:08 lauty95