Windows QField : File Locking prevents synchronization

[HeatherHillers]

What is the bug or the crash? What were your expectations and what actually happened?

When synchronizing projects in Windows QField.exe, I often have problems with file locking that prevent synchronisation and require deletion and reloading of the project to proceed. I have isolated one reproducable case. This appears to be tied to the development of the plugin, not to development of the data or the project.

Steps to reproduce the issue

1. clone http://github.com/HeatherHillers/qfield_vegetation_monitoring to ${ROOT}

2. create a project directory next to it: ${ROOT}/qfield_project_demo_1

3. Copy the project files and project data (Do Not Copy the Plugin yet!) to ${ROOT}/qfield_project_demo_1

   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/demo1_hello.qgs
   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/demo1_hello.qgs.png
   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/demo1_hello_attachments.zip
   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/vegetation.gpkg

4. Open the project in QGIS.

   • you should see just a single layer (plots) with a few points in it.

5. Upload the project to your QFieldCloud account using the QField Sync Plugin.

6. Load the project in the Windows QField -This should be successful.

7. Copy the plugin files to ${ROOT}/qfield_project_demo_1

   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/demo1_hello.qml
   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/components
   • ${ROOT}/qfield_vegetation_monitoring/demo1_hello/components/d1_plugin_component.qml

8. Reload the Plugin in QGIS

9. Resync the Plugin in QField Sync

10. Synchronise your Plugin in QField

You will (hopefully) see the following error:

So, this happens as a result of adding the plugin files to the project. The plugin is just a hello world.

Fix the problem by:

1. Deleting the local project in QField.exe and downloading it fresh.

Further investigation:

• After taking the above steps to fix the problem, I tested updating the data. I added a point to the layer in qgis and performed a sync. This update was able to sync in QField without error.
• I then tested a project file update: I changed the color of the points. ** This also triggered the same lock on data.gpkg.**
• I then started the test protocol over again, to see if changing the style in the project before adding the plugin files would trigger the error:

1. I deleted the qfield local project, the qgis local project, and the qfield cloud project.
2. I recreated the project directory and repeated the steps to copy the project files and sync the project with qfield. success.
3. I added a point to the layer in qgis and synchronised. success.
4. I changed the color of the points and synchronised. The same data.gpkg lock occured.

Conclusion: The data.gpkg lock happens when:

• adding (and I think also updating) a project plugin
• changing a style on a geopackage layer in a project with no plugin.
• changing a style on a geopackage layer in a project with a project plugin.

Version

QField 3.7.9. QGIS 3.42.2

Operating system name

Windows

Operating system version

Windows 11

Reinstall QField

• [x] I have a fresh install of the latest QField version, but the problem persists.
• [x] Problem can be reliably reproduced, doesn't happen randomly.
• [ ] Problem happens with all files and projects, not only some files or projects.

Additional context

I cannot prove that it happens with all files and projects, but it has been a frequently recurring issue during development.

[nirvn]

@HeatherHillers , could you give this a try with a master build to see if it's still a problem? You can find the latest revision as a windows installer artifact here (https://github.com/opengisch/QField/actions/runs/19525358323).

[HeatherHillers]

@nirvn Thanks for picking this up.

The master link you sent me has the same problem. My test protocol and the debug output from the qfield terminal follow.

Protokoll:

1. Found a different Windows 11 computer with no qgis or qfield on it (I was having whitelisting problems with the dev exe.)
2. installed qgis 3.44 fresh from osgeo network installer
3. installed the Gwaii release fresh to confirm the problem persists in the new setup.
4. deleted the project from qfield.cloud
5. uploaded the project (no plugin, just project) to the cloud
6. synchronised in gwaii, ok.
7. changed color of point layer in qgis, resynced
8. gwaii has data.gpkg error. Problem confirmed.
9. removed the stored project from my qfield profile
10. removed the project from qfield.cloud
11. Installed qfield master from the link you provided, under a program files\QField_dev to ensure fresh install.
12. ran qfield master
13. uploaded project in qgis as new project in qfield.cloud.
14. synced to qfield master. Ok.
15. changed layer color in qgis, saved, resynced.
16. qfield master reports the same data.gpkg error.

Output to QField master Terminal:

Unfortunately, the output ends with the first successfull synchronisation. There is no additional output as a result of the failed synchronisation.

[sentry] INFO using database path "C:\Users\Heather\AppData\Roaming\OPENGIS.ch\QField.sentry-native" [sentry] DEBUG starting transport [sentry] DEBUG starting background worker thread [sentry] DEBUG starting backend [sentry] DEBUG background worker thread started [sentry] DEBUG processing and pruning old runs Proj path: "C:\Program Files\QField_dev\usr\share\proj" qt.qml.typeregistration: Invalid QML element name "Qgis"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "CoordinateReferenceSystem"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "QgsUnitTypes"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "QgsWkbTypes"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "FeatureIterator"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "GridAnnotation"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "GnssPositionInformation"; value type names should begin with a lowercase letter qt.qml.typeregistration: Invalid QML element name "QgsLocatorContext"; value type names should begin with a lowercase letter qrc:/qml/imports/Theme/QfProjectThumbnail.qml:52:5: QML QQuickImage: Failed to get image from provider: image://projects//Users/Heather/AppData/Roaming/OPENGIS.ch/QField/cloud_projects/heather/1d15f896-c756-4866-92b6-160231d7eb6f/solar_qfield.qgs qt.multimedia.ffmpeg: Using Qt multimedia with FFmpeg version 7.1.2 LGPL version 2.1 or later qt.qpa.mime: Retrying to obtain clipboard. qrc:/qml/QFieldCloudProjectDetails.qml:234: TypeError: Cannot read property 'hasError' of null qrc:/qml/QFieldCloudProjectDetails.qml:56:9: QML QQuickImage: Cannot open: file://c/Users/Heather/AppData/Local/Temp/gtAdBb.PNG "AppInterface loading file: C:/Users/Heather/AppData/Roaming/OPENGIS.ch/QField/cloud_projects/heatherhillers/d0dc09a9-157c-4353-889c-2a93770fbcce/demo1_hello_qfield.qgs" DeltaFileWrapper::getLocalPkAttribute: vl->primaryKeyglAttributes()= QList(0) pkAttrs= QList(0, 1) DeltaFileWrapper::getLocalPkAttribute: pkAttrName= "fid_1" pkAttrIdx= 0 DeltaFileWrapper::getSourcePkAttribute: getting pkAttrNamesAggr= "fid" with type= QString DeltaFileWrapper::getSourcePkAttribute: pk attrs pkAttrNames= 1 DeltaFileWrapper::getSourcePkAttribute: pk pkAttrName= "fid" with index= 1 DeltaFileWrapper::getLocalPkAttribute: vl->primaryKeyglAttributes()= QList(0) pkAttrs= QList(0, 1) DeltaFileWrapper::getLocalPkAttribute: pkAttrName= "fid_1" pkAttrIdx= 0 DeltaFileWrapper::getSourcePkAttribute: getting pkAttrNamesAggr= "fid" with type= QString DeltaFileWrapper::getSourcePkAttribute: pk attrs pkAttrNames= 1 DeltaFileWrapper::getSourcePkAttribute: pk pkAttrName= "fid" with index= 1

[HeatherHillers]

@nirvn I have found the trigger. I considered that it must be a problem with the project configuration.

1. I made a new project with just the point layer and the default configuration. This did not display the data.gpkg error. (Yay!)
2. I examined the differences between the new default configuration project and my demo1_hello project.
3. I found that the QField Configuration for demo1_hello was configured for online editing but the layers were configured for offline editing. A user configuration error (or is it? is there a use case for that?), but one that is pretty easy to make. All you have to do is remember to configure the layer for offline editing in the layer properties, but forget to tell the qfield configuration to prefer offline editing.

I tested that configuration, and it does seem to be the trigger of the error, although reproducing is a bit tricky.

1. If you create a fresh project with an offline error and an online project, and test sync: change layer style: sync, you will reliably see the error.
2. If you change the project setting to online and resync, with or without creating a new project, it will fix the error. The project will sync.
3. Once the project is correctly configured and syncing, if you try to reproduce the error by going back to the "bad" configuration, it will not reproduce the error. The project will sync. You have to start a fresh project to see the error again.

So, there is the trigger.

Should it be possible / what would be the use case for offline layers in a project that prefers online editing? Is there a use case where some layers are offline and some online? What does it mean that the project prefers online or offline editing: Does the project level configuration override the layer configuration or does it just govern the default settings of the layers?

Can we get either a solve of the data.gpkg error, preventing upload of a mixed configuration in QField Sync, or a better error message on synchronization.

[HeatherHillers]

Hi @nirvn! Today I realized that I was not working with the latest version of QField Sync Plugin. I was embarrassed, but I thought that had at least solved the problem. But that did not solve the problem. With QField Sync 4.17.3 and a freshly built qgis project I can reproduce the error. I just select "prefer Online layers" from the configuration menu, instead of it being a radio button.