scancode.io
scancode.io copied to clipboard
Report progress when doing matching in d2d step
When doing a JS and Java D2d on a large codebase, there is either little or no progress provided
2024-06-27 03:27:44.41 Step [match_directories_to_purldb] starting
2024-06-27 03:27:45.65 Matching 78,856 directories from to/ in PurlDB
2024-06-27 03:41:29.99 Progress: 10% (7886/78856) ETA: 7419 seconds (2.1 hours)
2024-06-27 04:00:04.20 Progress: 20% (15772/78856) ETA: 7754 seconds (2.2 hours)
2024-06-27 04:10:07.26 Progress: 30% (23657/78856) ETA: 5930 seconds (1.6 hours)
2024-06-27 04:30:06.73 Progress: 40% (31543/78856) ETA: 5612 seconds (1.6 hours)
2024-06-27 04:45:46.57 Progress: 50% (39428/78856) ETA: 4681 seconds (1.3 hours)
2024-06-27 05:03:54.35 Progress: 60% (47314/78856) ETA: 3846 seconds (1.1 hours)
2024-06-27 05:24:31.20 Progress: 70% (55200/78856) ETA: 3002 seconds (50.0 minutes)
2024-06-27 05:45:52.57 Progress: 80% (63085/78856) ETA: 2072 seconds (34.5 minutes)
2024-06-27 06:05:29.40 Progress: 90% (70971/78856) ETA: 1052 seconds (17.5 minutes)
2024-06-27 06:25:34.22 Progress: 100% (78856/78856)
2024-06-27 06:25:34.41 16,889 directories matched in PurlDB
2024-06-27 06:25:34.42 Step [match_directories_to_purldb] completed in 10670 seconds (3.0 hours)
for these I would like to see progress displayed at a fixed time interval, rather than based percentage completion.
and also
2024-06-27 06:25:34.42 Step [match_resources_to_purldb] starting
2024-06-27 06:25:36.44 Matching 201,361 .map, .js, .mjs, .ts, .d.ts, .jsx, .tsx, .css, .scss, .less, .sass, .soy, .class resources in PurlDB, using SHA1
for these I would like to see some progress displayed at a fixed time interval, with percentage completion. Nothing is displayed at all