kepler icon indicating copy to clipboard operation
kepler copied to clipboard

Casual import error

Open banditopazzo opened this issue 1 year ago • 0 comments

Sometimes I get the following errors with import_nist commands:

[2023-07-10T20:30:44Z INFO  domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2009.json.gz to /data/nvdcve-1.1-2009.json.gz ...
[2023-07-10T20:30:44Z INFO  domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2009.json.gz to /data/nvdcve-1.1-2009.json ...
Error: could not extract /data/nvdcve-1.1-2009.json.gz

Caused by:
    invalid gzip header
[2023-07-10T20:30:44Z INFO  domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2010.json.gz to /data/nvdcve-1.1-2010.json.gz ...
[2023-07-10T20:30:45Z INFO  domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2010.json.gz to /data/nvdcve-1.1-2010.json ...
Error: could not extract /data/nvdcve-1.1-2010.json.gz

Caused by:
    invalid gzip header
[2023-07-10T20:30:45Z INFO  domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2011.json.gz to /data/nvdcve-1.1-2011.json.gz ...
[2023-07-10T20:30:45Z INFO  domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2011.json.gz to /data/nvdcve-1.1-2011.json ...
Error: could not extract /data/nvdcve-1.1-2011.json.gz

Caused by:
    invalid gzip header

If I try to launch the import again it finds the previous downloaded files and output this instead:

[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] found /data/nvdcve-1.1-2020.json
[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2020.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2020.json

Caused by:
    EOF while parsing a value at line 1 column 0
[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] found /data/nvdcve-1.1-2021.json
[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2021.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2021.json

Caused by:
    EOF while parsing a value at line 1 column 0
[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] found /data/nvdcve-1.1-2022.json
[2023-07-10T20:43:43Z INFO  domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2022.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2022.json

Caused by:
    EOF while parsing a value at line 1 column 0

The workaround is to delete each "corrupted" data file and start the relative year import again

banditopazzo avatar Jul 10 '23 20:07 banditopazzo