kepler
kepler copied to clipboard
Casual import error
Sometimes I get the following errors with import_nist
commands:
[2023-07-10T20:30:44Z INFO domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2009.json.gz to /data/nvdcve-1.1-2009.json.gz ...
[2023-07-10T20:30:44Z INFO domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2009.json.gz to /data/nvdcve-1.1-2009.json ...
Error: could not extract /data/nvdcve-1.1-2009.json.gz
Caused by:
invalid gzip header
[2023-07-10T20:30:44Z INFO domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2010.json.gz to /data/nvdcve-1.1-2010.json.gz ...
[2023-07-10T20:30:45Z INFO domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2010.json.gz to /data/nvdcve-1.1-2010.json ...
Error: could not extract /data/nvdcve-1.1-2010.json.gz
Caused by:
invalid gzip header
[2023-07-10T20:30:45Z INFO domain_db::cve_sources] downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2011.json.gz to /data/nvdcve-1.1-2011.json.gz ...
[2023-07-10T20:30:45Z INFO domain_db::cve_sources::nist] extracting /data/nvdcve-1.1-2011.json.gz to /data/nvdcve-1.1-2011.json ...
Error: could not extract /data/nvdcve-1.1-2011.json.gz
Caused by:
invalid gzip header
If I try to launch the import again it finds the previous downloaded files and output this instead:
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] found /data/nvdcve-1.1-2020.json
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2020.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2020.json
Caused by:
EOF while parsing a value at line 1 column 0
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] found /data/nvdcve-1.1-2021.json
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2021.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2021.json
Caused by:
EOF while parsing a value at line 1 column 0
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] found /data/nvdcve-1.1-2022.json
[2023-07-10T20:43:43Z INFO domain_db::cve_sources::nist] reading /data/nvdcve-1.1-2022.json ...
Error: failed to parse cve file from /data/nvdcve-1.1-2022.json
Caused by:
EOF while parsing a value at line 1 column 0
The workaround is to delete each "corrupted" data file and start the relative year import again