qfieldcloud icon indicating copy to clipboard operation
qfieldcloud copied to clipboard

Changes not applied - Unable to find feature

Open miili opened this issue 1 year ago • 29 comments

A bunch of changes failed with Unable to find feature and could not be applied.

The changes came in in small chunks (3 deltas). Applying the change again does fail with the same error.

This is related to https://github.com/opengisch/qfieldcloud/issues/324

miili avatar Mar 16 '23 12:03 miili

Can you give me any details about username/projectname and ideally, delta ids that you want to know more about. Also what is the data format of the layers, GeoPackage, PostGIS?

suricactus avatar Mar 16 '23 15:03 suricactus

Dataformat is GeoPackage. Change eff4a084-aef5-4b9b-8efa-321fb23f0ab0 Project Eifel-Large-N/Re-Deployment

miili avatar Mar 16 '23 15:03 miili

@suricactus when you see patch dd6fff05-b9ad-47c8-a253-8ae54c405c10 the sourcePk and localPk is the same. However the sourcePk should be 383.

Is this due to an outdated QField client app?

miili avatar Mar 16 '23 16:03 miili

I have 8 pages of changes here and none of them were properly applied!

Status says "applied" but they were not applied in the GeoPackage downstream. Yesterday's work had differing sourcePk and localPk.

What is going on? This is pretty critical! 💣

miili avatar Mar 16 '23 16:03 miili

Now that QfieldCloud is offered as a paid service (which I am 💯 supportive of) bugs like this should be ironed out!! I see hundreds of changes from today which appear applied, but are not. We were confident to use qfieldcloud in production, but I am getting a little nervous now.

miili avatar Mar 16 '23 17:03 miili

All that was changed in-between the working versions was the styling in the project file.

miili avatar Mar 16 '23 17:03 miili

Hi, @miili . This is definitely something that is of high importance.

when you see patch dd6fff05-b9ad-47c8-a253-8ae54c405c10 the sourcePk and localPk is the same. However the sourcePk should be 383.

How do you know the source PK is wrong in this one? Adding a bit more context will help finding the root cause.

EDIT: looking at the delta contents, I assume the sourcePk should not be 383, but 219?

Is this due to an outdated QField client app?

Should not be the case, can you please share the version you are using?

suricactus avatar Mar 16 '23 17:03 suricactus

Mostly 2.7.1 is used in the field by the different teams, but not by all teams in the field. I suspect it is a problem with the project.

The matching sourcePk seems to be a common issue: https://github.com/opengisch/qfieldcloud/issues/324#issuecomment-1385168652

miili avatar Mar 16 '23 17:03 miili

Seems there is some root problem might be from QgsOfflineEditing core plugin. Will keep you updated what are our findings.

suricactus avatar Mar 16 '23 17:03 suricactus

See for example these features:

  • Only the diamond shaped features should have been modified: Move feature, update attributes
  • (probably due to bad sourcePk another feature was moved and edited

This already is a big mess. We probably have to move back to paper for the week to come 😞

image

miili avatar Mar 16 '23 17:03 miili

Hi,

Maybe I have the same problem as yours :

image

I wrote a ticket and open a discussion about this problem because as miili, it's quiet critical because I can't put Qfield in production anymore... Project URL : https://app.qfield.cloud/a/SOPRECO/CCGP_CBT_3-0/ Discussion : https://github.com/opengisch/QField/discussions/4015

Without talking about this other problem : https://github.com/opengisch/QField/discussions/4007 Both cumulated, it's harder and harder to be confident, happily, I'm postive attitude ! :-)

GlaDal avatar Mar 16 '23 17:03 GlaDal

I uploaded the GeoPackage from the beginning of the working day, re-applied all changes from the day (~500 deltas), and reproduced the same bad state. See job b6e81eef.

Something is really odd!

miili avatar Mar 16 '23 21:03 miili

After more investigation, I am pretty confident that the localPk and sourcePk are referring to the same fid_1, where instead sourcePk should point to fid. Subsequently the wrong feature pointing to fid_1 is modified and everything is messed up.

How to access the API and download all the changes?

miili avatar Mar 16 '23 22:03 miili

@miili thanks for doing the deep parallel investigation. We have identified the issue as I have posted 4 hours ago. All the team resources are focused on fixing this issue, so expect results very soon.

We know how important is data for our users and we are looking for a solution where you do not need to download the changes and apply them yourself. Since the issue escalated a few hours ago, it will take a bit of time to be evaluated and implemented, but I completely understand your urgency. Please give us a window of a few hours (midnful it's midnight in the teams timezone) and we will provide you with all the information you need.

After we make sure we prevent this from hapening again, we can also provide you with all your changes.

Hope makes you feel a bit better, apologies for the inconvenience.

suricactus avatar Mar 16 '23 22:03 suricactus

Great, thank you!

I got all the deltas from the API and will use the day to apply changes locally. If needed I can share the scripts.

miili avatar Mar 17 '23 08:03 miili

Thanks for the heads up. Will keep you posted on our side too.

suricactus avatar Mar 17 '23 08:03 suricactus

Hi @miili we we've clearly identified what the issue is and have assessed the impacted projects. The issue seem to be related to a very specific use case and we have 23 users affected by the issue out of 45K users. We are now investigating what project configuration leads to the issue. we'll keep you updated as soon as we've more information. cheers Marco

mbernasocchi avatar Mar 17 '23 15:03 mbernasocchi

Thanks for the heads up. What exotic configuration did we use for our project?

miili avatar Mar 20 '23 18:03 miili

Hello together, My Projects are also affected of this bug and i can reproduce it too. A Workaround to fix this is to delete and download the Project again.

If you need more Information or betatesting (i have a selfhosted cloud currently), where i could apply a patch e.g.

stcz avatar Mar 23 '23 12:03 stcz

Can you please send me an email at [email protected] for additional information?

suricactus avatar Mar 23 '23 15:03 suricactus

Perhaps I am also involved... All the 'not applied' changes are because of "unable to find feature". I tried to re-apply them, but that didn't work out.

afbeelding

Apart from that, some features of collaborators and myself where registered twice. I don't understand how this could happen in the field.

afbeelding

BoswachterMarc avatar Apr 06 '23 18:04 BoswachterMarc

Here is another user who has the same problems when synchronizing. The error "Unable to find feature" appears many times, but this entity does exist in the project.

kiri03 avatar May 15 '23 16:05 kiri03

When one of my organization's editors makes changes to the project logs I get the error: "Unable to find feature" and the patch shows up as "Not_applied", but the feature is still in the project. It's a bit frustrating since we're paying for a service that doesn't seem to be working as it should.

From what I have been able to verify the error occurs when trying to sync changes made on the phone with a theme loaded that is different from the original project that was uploaded to QFieldCloud. In my case, I have a project in QGIS with a theme loaded by default that I upload to the QField cloud. If once successfully uploaded, I download it on the phone and change any record, the changes are applied correctly on the server. But if on the phone I change the default theme to another one that is already defined and then I make a change in the same record as before, when trying to upload the changes to the server an "Unable to find feature" error occurs.

kiri03 avatar May 15 '23 19:05 kiri03

I don’t know if it has anything to do with it, but the theme that was producing the errors refers to a symbology that has embedded SVGs in the project and they are rotated with the value that a specific field has.

kiri03 avatar May 16 '23 09:05 kiri03

Same problem here... Several changes not applied because "Unable to find feature". I checked and the features are there (id are matching).

Any workaround before patching this ?

graymondon avatar Jun 07 '23 09:06 graymondon

Same problem here ! I put images of the related changes details and job details failures (I don't understand that message : "empty sourceLayerId from localLayerId" image image

I don't know if it's in relation with themes uses (See post here), I have in this project 6 differents themes. This same problem was posted here on March 19th.

Thanks for helping

Are we only 23 users to have this problem ?

GlaDal avatar Jun 14 '23 13:06 GlaDal

#https://github.com/opengisch/qfieldsync/issues/508

GlaDal avatar Jun 14 '23 14:06 GlaDal

I'm also affected: image

sth-HMS avatar Jan 04 '24 09:01 sth-HMS

{
  "conflicts": null,
  "delta_file_id": "4bd43c74-e88b-4ae8-b004-eaa91dbeeea8",
  "delta_id": "01042462-700b-45d6-8ba2-3453ccaa0361",
  "delta_index": 0,
  "e_type": "ERROR",
  "feature_pk": "{9a8a24b4-6fbe-4a1d-aca2-05425f951cb8}",
  "layer_id": "test_01c1f56e_701b_44fc_b85a_5a400914d8be",
  "method": "patch",
  "modified_pk": null,
  "msg": "Unable to find feature",
  "provider_errors": null,
  "status": "status_apply_failed"
}

The problem persists and apparently without any reproducible logic. Unfortunately, it makes data collection unreliable. Is there any news?

gisuser0 avatar Jul 31 '24 13:07 gisuser0