open-balena-api icon indicating copy to clipboard operation
open-balena-api copied to clipboard

TransactionClosedError: Transaction has been rolled back while provisioning

Open bartversluijs opened this issue 3 years ago • 3 comments

Hi,

I'm trying out the most recent release of the API (v0.143.0), but while provisioning a device (Raspberry Pi 4, BalenaOS 2.80.5+rev1 (development), supervisor v12.8.7) I get the following error in the supervisor:

[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":"<html><body><h1>503 Service Unavailable</h1>\nNo server is available to handle this request.\n</body></html>\n","stack":"ApiError: <html><body><h1>503 Service Unavailable</h1>\nNo server is available to handle this request.\n</body></html>\n\n    at Object.<anonymous> (/usr/src/app/dist/app.js:10:465624)\n    at Generator.next (<anonymous>)\n    at fulfilled (/usr/src/app/dist/app.js:10:1950372)\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"}}

Checked the logs of the openBalena API, and I got the following error:

Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]: TransactionClosedError: Transaction has been rolled back.
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at env.DEBUG.rejectFn (/usr/src/app/node_modules/@balena/pinejs/src/database-layer/db.ts:132:27)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at PostgresTx.closeTransaction (/usr/src/app/node_modules/@balena/pinejs/src/database-layer/db.ts:230:36)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at PostgresTx.rollback (/usr/src/app/node_modules/@balena/pinejs/src/database-layer/db.ts:302:9)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at Object.transaction (/usr/src/app/node_modules/@balena/pinejs/src/database-layer/db.ts:384:15)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at runMicrotasks (<anonymous>)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at processTicksAndRejections (internal/process/task_queues.js:95:5)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]:     at register (/usr/src/app/src/features/device-provisioning/register.ts:62:20)
Jun 25 10:09:24 openbalena-api-568b5c6956-z88vb api[1061]: 2021-06-25T10:09:24.950Z 10.110.0.12 a/6 POST /device/register 401 20.664ms -

I've attached a log file with some more logs below, as I don't know what's relevant to the issue itself. Tried downgrading to v0.138.7 and provisioning went well.

[email protected]


Currently testing until what version all goes well...

bartversluijs avatar Jun 25 '21 10:06 bartversluijs

Tested it out, because I saw a change that has something to do with the provisioning in the changelog, and it seems like PR #657 causes this issue.

Everything works fine till v0.139.0, but since v0.139.1, the error occurs.

Hope this will point someone in the right direction!

bartversluijs avatar Jun 25 '21 10:06 bartversluijs

Some more info, this is the body sent to the endpoint:

{
  application: 1,
  uuid: '251eac95e0a28b7727d48625d27f7914',
  device_type: 'raspberrypi4-64',
  api_key: 'f68dd08405bbb34ee45e3d04ee2e7b64',
  supervisor_version: '12.8.7',
  os_version: 'balenaOS 2.80.5+rev1',
  os_variant: 'dev',
  mac_address: 'DC:A6:32:57:E4:FE 72:28:DB:FA:14:68'
}

bartversluijs avatar Jun 28 '21 09:06 bartversluijs

The function getOSReleaseResource crashes in the 'postHostappHook'. I don't really know what this does, so I'm hoping someone can investigate what's wrong with this request. Just as some info, it's a clean openBalena instance with just 1 app and 1 release that belongs to that app.

The request that's being executed is:

Parsing GET /resin/release?$select=id,belongs_to__application&$filter=(status eq 'success') and (belongs_to__application/any(a:(a/is_for__device_type eq 61) and (a/is_host eq true))) and ((release_tag/any(rt:(rt/value eq '2.80.5%2Brev1') and (rt/tag_key eq 'version'))) and (release_tag/any(rt:(rt/value eq 'development') and (rt/tag_key eq 'variant'))))

https://github.com/balena-io/open-balena-api/blob/b7545c67c7f85509f1c1a559eb297ac3a0af6b27/src/features/hostapp/hooks/target-hostapp.ts#L105

bartversluijs avatar Jun 28 '21 09:06 bartversluijs