Timestamp issues displaying in UX

[gaelmuller]

Describe the bug

Two issues were introduced in https://github.com/google/timesketch/commit/7fd82062a8ce031270a31bde8e8acaf10c0b5798:

• The formatTimestamp filter has a conversion bug for timestamps in seconds (* 1000000 should be * 1000 to convert seconds to milliseconds). This results in odd timestamps being displayed in the UI when using this timestamp format.
• The timestamp field is now used to display the Date & Time information but this field is not required (see https://github.com/google/timesketch/blob/master/docs/learn/create-timeline-from-json-csv.md for the documented list of required fields)

I could probably do a PR to fix this but I'm not sure which way this should be fixed: revert back to using datetime or fix the helper and add timestamp to the required fields ?

[gaelmuller]

I think there is also an issue with the way timestamps are displayed:

{{ event._source.timestamp | formatTimestamp | moment("utc", datetimeFormat) }}

If I understand this correctly, this is considering the browser's time zone to convert the date extracted from the timestamp to UTC, except the timestamp should already be in UTC. This results in wrong datetimes being displayed.

[berggren]

Hi, thanks for this report! Inline:

Describe the bug

Two issues were introduced in 7fd8206:

• The formatTimestamp filter has a conversion bug for timestamps in seconds (* 1000000 should be * 1000 to convert seconds to milliseconds). This results in odd timestamps being displayed in the UI when using this timestamp format.

good catch - I'll look into it.

• The timestamp field is now used to display the Date & Time information but this field is not required (see https://github.com/google/timesketch/blob/master/docs/learn/create-timeline-from-json-csv.md for the documented list of required fields)

The timestamp is not required because it will be generated from the datetime field if missing.

I could probably do a PR to fix this but I'm not sure which way this should be fixed: revert back to using datetime or fix the helper and add timestamp to the required fields ?

So, no change needed here as the logic to create the timestamp if missing already exists.

[berggren]

I think there is also an issue with the way timestamps are displayed:

{{ event._source.timestamp | formatTimestamp | moment("utc", datetimeFormat) }}

If I understand this correctly, this is considering the browser's time zone to convert the date extracted from the timestamp to UTC, except the timestamp should already be in UTC. This results in wrong datetimes being displayed.

It is actually the other way around :) If we don't use the UTC function here it will use the local time when displaying the time (based on the timestamp)

[gaelmuller]

I think there is also an issue with the way timestamps are displayed:

{{ event._source.timestamp | formatTimestamp | moment("utc", datetimeFormat) }}

If I understand this correctly, this is considering the browser's time zone to convert the date extracted from the timestamp to UTC, except the timestamp should already be in UTC. This results in wrong datetimes being displayed.

It is actually the other way around :) If we don't use the UTC function here it will use the local time when displaying the time (based on the timestamp)

Here is what I'm seeing:

Removing "utc" fixes it for me :thinking:

[jaegeral]

Is there an update on this @berggren ?

[jaegeral]

Friendly ping @berggren

[jaegeral]

Has not been touched in a while and we moved the UI in the meantime, closing this one out