Time saved 1 hour out from input time

[jpnunn]

Describe the issue

For example 1 when time is entered in a form it is synced and then it appears that the time is 1 hr ahead. For example 2 The datetime is input but when shown in the display of a value relation it comes out an hour earlier.

Reproduction steps

Example 1 When input: When synced:

Example 2: When input: When viewing the value relation list before syncing: Once synced both the input value and the display value shown above are shifted an hour ahead.

Add a simple sample project if it requires a specific configuration

Expected behavior

In theory the time should stay the same when input, when synced , or in the value relation list

Observed behavior

The input time appears correct. Then when sycned it appears out by 1 hour ahead.

QGIS (please complete the following information)

• OS: Windows 11
• QGIS Version 3.34.8
• QFieldSync Version 4.10.1

QField (please complete the following information)

• Device: Samsung tablet
• OS: ?
• QField version: Should auto update to latest. Happening on mutiple devices / users

Additional information

• Problem started happening recently, didn't happen in an older version of QField: Unsure
• Problem can be reliably reproduced, doesn't happen randomly: Yes
• Problem happens with all files and projects, not only some files or projects: Yes

[If the problem happened with QFieldCloud, please add your username and project name.]

[glavocat]

Here in France, I have the same problem since the last version of QField.

QField (please complete the following information) Device: Samsung phone OS: Android 13 QField version: Auto update to latest.

[nirvn]

@jpnunn , could you share a sample project to replicate this issue? Thanks

[jpnunn]

Hi @nirvn I have setup a project here you can use called 11111_QFIeld_Time_Test . Try editing the SessionDetails_table, and look at the start time and created and edited dates. ALso within that table there is a tab for secondary species that has a relation in it. Try adding rows into that

I also opened an issue on the QFieldCloud as i belive it might be cloud that is causing one of the problems. One i still think is caused by the QField app where the value relation list display is reduced by 1 hour : https://github.com/opengisch/QFieldCloud/issues/1045

[VxTedxV]

not sure but may be also related to #3988 ?

[9ls1]

I have a similar problem: Start time and end time are saved differently in QGIS and QField!?! I have a geopackage file with a point layer "arter" containing two fields "start" and "slutt" (type DateTime) that both are set as widget type Date/Time (ISO Date Time format) in the Attributes Form. Field "start"

Field "slutt" (Note the default value)

Based on "start" and "slutt", the time used is calculated and saved in field "tidsforbruk" as text with default value like "if("ferdig"=1,format_number(minute(age("slutt","start")),0),null)"

In QGIS: A point 6 ("kprutenr") is recorded (the times correspond to my local time UTC + 2 h, as shown on my computer) Then I click Save edits, and the times ("start" and "slutt") are saved as UTC. This is OK, the (relative) time consume will be 1 minute.

I transfer the project to QFieldCloud, download it in QField on the tablet. The times for the existing point 6 are shown as local time (e.g. "start" 15:27:11+GMT+02:00 and "slutt" 15:28:03+GMT+02:00, "tidsforbruk" 1 minute). I add a new point 3 ("kprutenr") and the times and time consume are shown correctly in QField. I push the recording to the cloud and download to QGIS. I open the attribute table and see that "start" is saved included + 2 hours (equals to my local time - as shown on the PC-clock) while "slutt" is saved without the 2 hours (even though both say West-Europe (summer time). This resulting in negative time consume of -119 minutes (in stead of the correct 1 minute).

Any explanations?

I'm using QGIS 3.42.0, QFieldSync 4.13.1, self-hosted QFieldCloud and QField 3.5.4.

[9ls1]

@nirvn Test dataset: demo.zip

[VxTedxV]

Any possible link to #3988 ?

[9ls1]

@VxTedxV Maybe, but in "my" case the "start"-attribute and the "slutt"-attribute behave (sets/calculates) the time-value differently only in kprutenr 3 after transferring from the cloud and opened (displayed) in QGIS/attribute table. This is really crucial since the calculated time consume will be wrong.

[SeqLaz]

Hi @9ls1 ,

This happen because GeoPackage always uses UTC datetime by default to store date, and date time.

https://gdal.org/en/stable/drivers/vector/gpkg.html#dataset-creation-options

One workaround is to add two new fields for each of the datetime field required, one of type Integer 64 (to store the epoch) and another of type Text (This to store the human readable representation of datetime). On the integer type, use the expression epoch(now()) or the epoch("datetime_field") , and on the Text field, use the expression to_string(format_date(datetime_from_epoch(epoch("epoch_field")), 'yyyy-MM-dd HH:mm:ss+T')). This way, you will save the information with the format of your device and not in UTC.

As for the calculation of the minutes, use a decimal type (not a datetime type field see how is stored when used a datetime for the duration), and use datetime_from_epoch(epoch("field_with_the_epoch")) expression to return a datetime for the calculation only, for example minute(age(datetime_from_epoch(epoch("endtime_epoch")), datetime_from_epoch(epoch("starttime_epoch"))))

[9ls1]

@SeqLaz Thanks! I'll try it asap when finished with other, new QField-projects – QField is getting very popular at my institute, so very busy at the moment. 👍

[9ls1]

@SeqLaz I have now tried your approach with two fields for the

• start time (field "startepoch" as integer64 and default value like epoch(now()) and field "start" as text and defualt value like to_string(format_date(datetime_from_epoch("startepoch"), 'HH:mm:ss')))
• end time (field "sluttepoch" as integer64 with default value (and apply on update) like if("ferdig"=1,epoch(now()),null) and field "slutt" as text with default value like (and apply on update) to_string(format_date(datetime_from_epoch("sluttepoch"), 'HH:mm:ss'))). The end-time will be captured when the user check "ferdig" (eng. finished), i.e. "ferdig" is given the value 1.

and calculating the duration (field "tidsbruk_min" as Double) based on start epoch and end epoch with default value (and apply on update) like if("ferdig"=1,format_number(minute(age(datetime_from_epoch("sluttepoch"), datetime_from_epoch("startepoch"))),0),null).

This works nice in QGIS and in QField "separately".

However, after finished in QField, pushing my recordings from QField to QFieldCloud and synchronizing (current cloud project) back to QGIS, the fields "sluttepoch" and "slutt" are changed to correspond to the "synchronize time"?!? And the duration is recalculated (and will be longer than the one calculated in QField during the field work)! The fields "startepoch" and "start" are unchanged (i.e. the original time captured), only "sluttepoch", "slutt" and "tidsforbruk_min" are changed.

In QField, kprutenr 8 and 7:

In QGIS after synchronize sluttepoch and duration are changed for rutekpnr 8 and 7:

What might be wrong? @nirvn: Is it QField synchronize that cause the time-values to change? I'm using QGIS 3.42, QField 3f66f5 v.3.5.5 with QField Sync 4.14.0 on self-hosted QFieldCloud v0.26.2 (it might be v0.26.0 or v0.26.1).

demo.zip The point layer "arter" contains the "time-fields".

[mihalruetimann]

Same issue here as @jpnunn:

In the mean time, time difference is 2 h not 1 h , probably due to change from winter (MEWT) to summertime (MEST). Happens only in QField App, not in QGIS.

QGIS OS: MAC OS Sequoia 15.5 QGIS Version 3.40.5-Bratislava QFieldSync Version 4.13.1

QField testet on diffrent devices (a and b) Device: a) Samsung Galaxy Tab Active3 b) iphone 12mini OS: a) Android 13 b) iOS 18.4.1 QField version: a) 3.5.3 - Fangorn b) 3.5.4 - Fangorn, also 3.6.0

Happening on mutiple devices / users Problem started happening after updating QField, didn't happen in an older version of QField: 3.2.2. Problem can be reliably reproduced, doesn't happen randomly: Yes Problem happens with all files and projects, not only some files or projects: Yes

After sync in QGis the time is 14:00, should be 16:00.

Happy for any help or improvement.

[mihalruetimann]

Hi again,

The (or part of the) underlying problem is a different behavior of QGis and QField: Layer with Field with type "Date & Time", default now() In QGis: first appearance when adding new point feature: time in local time, but as soon as feature is saved time is in UTC. In QField: time in local time, also after saving point.

Attached a simple test-project. As before, any help is appreciated.

Qfield_Generic_1.zip

[nirvn]

@mihalruetimann , thanks for the test case, it's helpful. I've looked into your project, and what I see is as follow:

• the QField date[time] editor widget shows the date and time in the local timezone
• when saving to the geopackage, the time is actually saved in UTC (looking at the raw data using sqlite)
• when viewing the data in QField through its feature form, the label shows the preexisting date and time data from features in the local timezone.

So as far as I can make it, the data itself is always saved in UTC, but QField shows it back into local timezone within the feature form. Does that sound right to you?

[mihalruetimann]

@nirvn Hi, thank you for looking into it. Yes, that sounds right. So probably this is just a matter of how it is shown, makes it tricky to understand whats happening, but ok. I think what we would need is, that there would be a way to tell QField (or also QGIS) that i want to add points without timezone. Geopackage saves datetime following the ISO8601. Following this ISO, at the end of the timestamp is a timezone mark, e.g. Z for UTC "2007-12-24T18:21Z", but following the ISO, it should be possible to give a timestamp without timezone. I hope i understood that correctly.

[nirvn]

@mihalruetimann , QGIS hardcodes UTC when saving to geopackage for datetime fields. That's why oddly enough it might just be easier to go for a string field there.

[gdt]

It strikes me that the root cause is confusion about display timezone. Storing as a number since epoch in UTC, or as ISO8601 in UTC in the database seems like obviously the right thing to do, and making that more complicated seems like a lot of cognitive load and work and is fundamentally a workaround.

Without experimenting, it seems obvious that qgis and qfield both should accept and display times in local time, and store as UTC. Just like every other unix program. If the user changes their timezone, that should change the display.

(It might help to put the timezone abbrev in the display, too.)