TTL configured，but vid still can be found after expiration,after compaction looks normal

[shanlai]

version:

steps

1. execute ngql:

 CREATE TAG IF NOT EXISTS ttl01 (a timestamp);
 ALTER TAG ttl01 TTL_COL = "a", TTL_DURATION = 60;
 INSERT VERTEX ttl01(a) VALUES "ttl01_01":(now());
 match (v:ttl01) return v limit 1;
 submit job compact
 match (v:ttl01) return v limit 1;

Execution plan of nGQL: match (v:ttl01) return v limit 1;

2. The problems found are as follows:

• tag: TTL configured，but vid still can be found after expiration,after compaction looks normal，edge same as tag

[critical27]

Which version is your kernel? 3.1 enterprise version?

[shanlai]

3.1.0

[HarrisChu]

for issue 2, refer https://github.com/vesoft-inc/nebula/issues/3958 we have two times:

• ::time(NULL);
• time::WallClock::fastNowInSec();

now() uses the first, and checkDataExpiredForTTL uses the second.

how to reduce:

1. start storage
2. change system time, e.g. date -s 04:12:00
3. execute query insert vertex l1(a) values "101":(now())

[critical27]

for issue 2, refer #3958 we have two times:

• ::time(NULL);
• time::WallClock::fastNowInSec();

now() uses the first, and checkDataExpiredForTTL uses the second.

how to reduce:

1. start storage
2. change system time, e.g. date -s 04:12:00
3. execute query insert vertex l1(a) values "101":(now())

👍

[critical27]

This need to verified after tagless vertex has been removed. It is expected behavior for the first issue

[jjsimps]

How come this was marked as won't fix? I have also encountered this issue. It's rather cumbersome to have to filter out invalid/empty results.

For example, when I iterate over the results, the entry shows up (previous deleted by TTL), but the are no tags associated with it. So when I try to access anything I get errors. Even though I filter specifically on a tag existing.

[wey-gu]

How come this was marked as won't fix? I have also encountered this issue. It's rather cumbersome to have to filter out invalid/empty results.

For example, when I iterate over the results, the entry shows up (previous deleted by TTL), but the are no tags associated with it. So when I try to access anything I get errors. Even though I filter specifically on a tag existing.

The tagless(bare tag vertex) support will be removed soon, and afterward this issue should be gone(or easier to be handled), right @critical27 @Sophie-Xie ?

[shanlai]

After the tagless vertex is removed, if the Vertex has multiple tags, the problem still exists

(root@nebula) [nba_ttl]> CREATE SPACE `ttl_test` (partition_num = 100, replica_factor = 1, charset = utf8, collate = utf8_bin, vid_type = FIXED_STRING(32), atomic_edge = false)
Execution succeeded (time spent 3127/17445 us)

Tue, 22 Nov 2022 10:26:03 CST

(root@nebula) [nba_ttl]> use ttl_test
Execution succeeded (time spent 996/14673 us)

Tue, 22 Nov 2022 10:26:13 CST

(root@nebula) [ttl_test]> CREATE TAG IF NOT EXISTS ttl01 (a timestamp);
Execution succeeded (time spent 2504/16683 us)

Tue, 22 Nov 2022 10:27:27 CST

(root@nebula) [ttl_test]>  ALTER TAG ttl01 TTL_COL = "a", TTL_DURATION = 10;
Execution succeeded (time spent 1647/16351 us)

Tue, 22 Nov 2022 10:27:27 CST

(root@nebula) [ttl_test]> CREATE TAG IF NOT EXISTS ttl02 (a timestamp);
Execution succeeded (time spent 2579/19678 us)

Tue, 22 Nov 2022 10:27:28 CST

(root@nebula) [ttl_test]>  INSERT VERTEX ttl02(a) VALUES "ttl01_01":(now());
Execution succeeded (time spent 984/18110 us)

Tue, 22 Nov 2022 10:27:36 CST

(root@nebula) [ttl_test]> INSERT VERTEX ttl01(a) VALUES "ttl01_01":(now());
Execution succeeded (time spent 1215/29407 us)

Tue, 22 Nov 2022 10:27:38 CST

(root@nebula) [ttl_test]> match (v:ttl01) return v limit 1;
+------------------------------------+
| v                                  |
+------------------------------------+
| ("ttl01_01" :ttl02{a: 1669084056}) |
+------------------------------------+
Got 1 rows (time spent 3435/19022 us)

Tue, 22 Nov 2022 10:27:52 CST

Cause of the problem: ScanVertices not filtering expired data

[shanlai]

fixed in https://github.com/vesoft-inc/nebula/pull/4578