HowardHinnant
About gps_time and a floor<days> issue
Hi Howard,
I'm working on a program that needs to format GPS time as part of a translator to export vehicle trajectory to Google Earth. I spent a couple of days learning to use the date.h and tz.h templates and ran into an issue with floor
Here is the problem: (not to preach to the choir) the natural time of the GPS constellation is expressed in weeks and seconds into the fractional week. High end receivers have microseconds or better time resolution. using date the problem is simple to solve but the hickup came when I tried to use floor. Here is the test program I'm working on.
`
using namespace std;
using namespace date;
using namespace chrono;
int main()
{
cout << "GPS Date conversion test!\n";
auto tp = system_clock::now(); // a chrono::time_point in UTC from system clock
auto dp = floor<days>(tp); // a time_point truncated to full days
auto mt = make_time(floor<seconds>(tp - dp));
cout << "Today's date: " << dp << endl;
cout << "Today's time: " << mt << endl;
/* PROBLEM: Convert GPS week and week seconds to UTC date and time */
/* SAMPLE DATA: August 13, 2022 17:41:01 MST (not exact seconds) */
int gpsweek = 2223;
double wkseconds = 2482.966; // GPS Week seconds as double
uint32_t wkmilli = 2482966; // GPS milliseconds as unsigned int 32 bits
auto wks = date::weeks{ gpsweek }; // OK
auto wsec = duration<double, milli>(wkseconds); // OK
auto wmsec = duration<uint32_t>(wkmilli); // ERROR
// Using the uint32_t version of weekmilliseconds produces the wrong date by decades
// isn't ratio supposed to be 1:1 ???
gps_time<milliseconds> gt{ round<milliseconds>(wks + wsec)}; // create gps time_point
auto utc = clock_cast<utc_clock>(gt); // convert to UTC
auto gd = floor<days>(gt);
auto gs = make_time(floor<seconds>(gt - gd));
cout << "GPS TIme: [" << gd << "] [" << gs << ']' << endl;
auto ud = floor<days>(utc);
auto ut = make_time(floor<seconds>(utc - ud));
cout << "UTC TIme: [" << ud << "] [" << ut << ']' << endl;
auto sys = clock_cast<system_clock>(gt);
auto sd = floor<days>(sys);
auto st = make_time(floor<seconds>(sys - sd));
cout << "SYS TIme: [" << sd << "] [" << st << ']' << endl;
cout << date::format("Formatted: [%F] [%T]\n", floor<seconds>(gt));
}
// ------------------------------------ Output
// GPS Date conversion test!
// Today's date: 2022-09-17
// Today's time: 02:29:59
// GPS TIme: [2022-08-14 00:00:00] [00:00:02] <-- error: gps_time output times after floor<days>()
// UTC TIme: [2022-08-13 23:59:33] [00:00:11] <-- error: utc_time outputs time after floor<days>()
// SYS TIme: [2022-08-13] [23:59:44] <-- OK casting to system_clock works as docuented.
// Formatted: [2022-08-14] [00:00:02] <-- OK format alwo works as documented with gps_time
`
I'm not sure where the floor<> error is coming from but fortunatelly the format function works fine.
Hi. I see two errors so far:
auto wsec = duration<double, milli>(wkseconds); // OK
I believe this should be:
auto wsec = duration<double>(wkseconds); // OK
The milli you have interprets the double as 2482.966 milliseconds. I believe from your comment you intend 2482.966 seconds (i.e. 2482 seconds plus 966 milliseconds).
This:
auto wmsec = duration<uint32_t>(wkmilli); // ERROR
should be:
auto wmsec = duration<uint32_t, milli>(wkmilli);
What you have means 2482966 seconds. Adding the milli changes the meaning to 2482966 milliseconds.
If these changes don't give you the expected results, let me know, and I'll look some more.
Hello again, I did the changes that you suggested as follows to test both versions:
auto wsec = duration<double>(wkseconds);
auto wmsec = duration<uint32_t, milli>(wkmilli);
gps_time<milliseconds> gt1{ round<milliseconds>(wks + wsec)};
gps_time<milliseconds> gt2{ round<milliseconds>(wks + wmsec) };
The result with gt1 is:
GPS TIme: [2022-08-14 00:00:00] [00:41:22]
SYS TIme: [2022-08-14] [00:41:04]
which is about rigtht for August 13, 2022 17:41:01 MST, [GMT-7] less 18 GPS leap seconds.
Apparently, with a floating point there is no need for a ratio, it seems to be ignored. Is duration always assuming seconds with a float?
The result with gt2 is actually way out of range:
GPS TIme: [1980-01-07 00:00:00] [17:08:39]
SYS TIme: [1980-01-07] [17:08:39]
This is GPS birth date + 1 day and 17 hours. I tried different ratio factors with the int version but none of them seems to give the correct value. The logic one is milli as you say, and it was the first one I tried, but as you see above, it is completely out of range.
Thanks for quick return! Guillermo
Apparently, with a floating point there is no need for a ratio, it seems to be ignored. Is duration always assuming seconds with a float?
The duration constructor that takes an arithmetic type does no arithmetic whatsoever. It simply stores the value given. So:
auto wsec = duration<double, milli>(2482.966);
stores 2482.966 in a double inside of a duration<double, milli>, and then has the semantics of 2482.966 milliseconds.
auto wsec = duration<double>(2482.966);
stores 2482.966 in a double inside of a duration<double>, and then has the semantics of 2482.966 seconds. The second template parameter of duration defaults to ratio<1, 1> which has the semantics of seconds.
This is GPS birth date + 1 day and 17 hours.
This is how I would code and print this:
cout << gps_seconds{} + days{1} + 17h << '\n';
Output:
1980-01-07 17:00:00
OK the logic behind the duration<> is clear now.
I made the comment about duration assuming seconds because I get, not identical but very close results for the sample data with this:
auto wsec = duration<double, milli>(2482.966); // SYS TIme: [2022-08-13] [23:59:44]
and this:
auto wsec = duration<double>(2482.966); // SYS TIme: [2022-08-14] [00:41:04]
only 41 minutes and change of discrepancy.
In contrast this:
auto wmsec = duration<uint32_t, milli>(2482966); // SYS TIme: [1980-01-07] [17:08:39]
may have the correct milliseconds semantics, but has an error of 40 decades!
In fact, it produces a completely incorrect result with the same data set, regardles of the ratio used.
It is obvious that handling of semantics for ints and doubles is different, maybe at the chronos level?
Thinking back into my problem, probably using TZ.h is an overkill, as I only need an epoc shift from GPS time to UTM or SYS time.
Best regards, Guillermo
UPDATE
Making this change we get the correct result:
auto wmsec = duration<int>(2483); // SYS TIme: [2022-08-14] [00:41:05]
Seconds are assumed, But oter ratios break the logic somehow.
In contrast this: auto wmsec = duration<uint32_t, milli>(2482966); // SYS TIme: [1980-01-07] [17:08:39] may have the correct milliseconds semantics, but has an error of 40 decades!
I've gotten lost. Can you show a minimal complete program demonstrating this?
I think that you were focusing on "how to print...".
Actually I'm reporting two problems that I ran into:
- applying duration to an int for values other than seconds is broken
- printing a gps_time or utm_time with
floor<days>still print time with zeros. It only works correctly with system_time. By working correctly I mean it only prints the date and suppress the zero time.
I simplified the test program and added ample comments to show both problems as clear as I can, see the attached source file. It would be interesting to compare the output in a linux system. gpstest.zip
Problem 1: Incorrect rsults using int for duration other than seconds: The correct result (in GPS time) shoud be: This: 2022-08-14 00:41:23
g2 = 1980-01-07 17:08:39
The problem here is overflow of the int specified for the rep. weeks::rep and I_ms.rep are both int. So wks + I_ms also has a rep of int. The un-overflowed value of wks + I_ms is 1,344,472,882,966ms, which overflows the 32 bit int. If you use duration<int64_t, milli> instead, you'll avoid this overflow and get the desired result.
Problem 2: floor
(gps_time) still prints time with zeros: floor (g1) : 2022-08-14 00:00:00
Correct, and converting to sys_days is a reasonable way to get the formatting you want. Another more direct way to control the formatting is with format:
cout << format("%F", g1) << '\n';
Output:
2022-08-14
Ops!
The un-overflowed value of wks + I_ms is 1,344,472,882,966ms, which overflows the 32 bit int. If you use duration<int64_t, milli> instead, you'll avoid this overflow and get the desired result.
That explains, it, I did the change and it works as it should. Thanks for the clarification.
Somehwere in one of your videos, I think it was the one about chronos, I recall hearing that what doesn't fit in a int gets converted to a long long. When that conversion happens whithout user intervention?
Abusing of your time, I have one last question: What is the correct way to convert the durationss to sys_days in order to do a simple epoc shift like this example from your documentation (I ajsted it to GPS start date):
using time_point = std::chrono::time_point<std::chrono::system_clock, std::chrono::milliseconds>;
inline time_point shift_epoch(time_point t)
{
return t + (sys_days(January / 6 / 1980) - sys_days(January / 1 / 1970));
}
I can compenstte for leap seconds too, but in the context of my appication it is not that important. The objective is to show a vehicle trajectory in Google Maps and if it is 18 seconds ahead or behind it is irrelevant.
Thanks again for your quick returns! Best regards, Guillermo
Somehwere in one of your videos, I think it was the one about chronos, I recall hearing that what doesn't fit in a int gets converted to a long long. When that conversion happens whithout user intervention?
Never really. When two durations are added (or a time_point and a duration), the library forms the std::common_type of the two reps, and uses that common type for both the intermediate computations and the final result. But in your case both reps were int and so the common type was also int.
Now you can use a class type as the rep. For example one could use SafeInt as the rep. Then chrono arithmetic will do whatever SafeInt arithmetic does (e.g. throw an exception on overflow).
Also, most people (including myself) tend to use the "built-in" durations: seconds, microseconds, etc, instead of the duration template directly. The built-ins require larger signed-integral reps for finer precisions in order to make overflow less likely. E.g. this would also work:
milliseconds I_ms{I_wkmilli};
This is required to use at least 45 bits under the hood and is typically a 64 bit signed integral rep. And when added to weeks, the common type rep is a signed 64 bit integral, so no overflow happens.
What is the correct way to convert the durationss to sys_days
If the source/destination is gps_clock or system_clock, the best way is to use clock_cast:
#include "date/tz.h"
#include <iostream>
#include <chrono>
using namespace std;
using namespace date;
using namespace chrono;
int main()
{
sys_seconds sys_tp = sys_days{September/17/2022};
auto gps_tp = clock_cast<gps_clock>(sys_tp);
auto sys_tp2 = clock_cast<system_clock>(gps_tp);
cout << "sys_tp.time_since_epoch() = " << sys_tp.time_since_epoch() << '\n';
cout << "gps_tp.time_since_epoch() = " << gps_tp.time_since_epoch() << '\n';
cout << "sys_tp2.time_since_epoch() = " << sys_tp2.time_since_epoch() << '\n';
}
Output:
sys_tp.time_since_epoch() = 1663372800s
gps_tp.time_since_epoch() = 1347408018s
sys_tp2.time_since_epoch() = 1663372800s
This does the epoch shift for you, including the leap seconds. If you want to implement such an epoch shift yourself, say for a custom clock, then do it the same way gps_clock does it, by implementing these static member functions: https://github.com/HowardHinnant/date/blob/master/include/date/tz.h#L2197-L2219.
gps_clock implemented to_utc and from_utc (conversion to and from utc_clock). But this enables clock_cast to perform conversions between gps_clock and every other clock that has enabled clock_cast. One could also enable this functionality by implementing to_sys and from_sys. That is, if any clock enables conversions to and from system_clock, or utc_clock, then clock_cast will run with that, and provide conversions to all clocks that implement either of these conversions.