Help with archive and distributive functions

[dgmorrisjr]

Hello, I am working in python 2.7 and have successfully installed the vulners package from PyPi.

I also created an api key with scope of api.

I've tested other examples from the readme page and they seem to work well. I can run the query and then print the results to see the resulting json data.

However, when I try to run the following:

vulners_api = vulners.Vulners(api_key="YOUR_API_KEY_HERE")
all_cve = vulners_api.archive("cve")

The call seems to hang... e.g. >>> all_cve = vulners_api.archieve("cve") and doesn't return the python prompt. I've let this sit for up to 5min or longer and eventually it seems to just timeout.

from reviewing the code, i'm expecting the all_cve object to be a zip file. I'm assuming that I could then write this to a file locally with something like:

vulners_api = vulners.Vulners(api_key="YOUR_API_KEY_HERE")
all_cve = vulners_api.archive("cve")
with io.open('cve.zip', 'w+b', -1) as file:
    file.write(all_cve)

or

vulners_api = vulners.Vulners(api_key="YOUR_API_KEY_HERE")
with io.open('cve.zip', 'w+b', -1) as file:
    file.write(vulners_api.archive("cve"))

would there be a preference to the write way to write the cve.zip file to disk?

[bmarsh9]

I also had the same problem. Looks like there is just too much data. Try this which worked for me. Basically just pulls down the data incrementally (which Vulners probably likes anyway...) (only works with python2 or else the vulners library throws a error).

Change the following variables to what you want:

start --> start pulling data from this date end --> end at this date delta --> delta between in days output_file --> output file name

import vulners
import json
import datetime
import time
import json

api_key="<HERE>"
vulners_api = vulners.Vulners(api_key=api_key)

start=datetime.datetime(2018, 1, 1)
end=datetime.datetime(2018, 2, 2)
delta=3
output_file="cve.json"

def download_cve(start,end):
    start=start.strftime("%Y-%m-%d")
    end=end.strftime("%Y-%m-%d")
    all_cve = vulners_api.archive("cve",start_date=start, end_date=end)
    with open(output_file, "a") as myfile:
        for obj in all_cve:
            myfile.write(json.dumps(obj)+"\n")

while start < end:
    next_delta = start+datetime.timedelta(days=delta)
    print("Getting data between: {} - {}".format(start,next_delta))
    #make api call here
    download_cve(start,next_delta)
    time.sleep(1)
    start = start+datetime.timedelta(days=7)
    start = next_delta
    #break
print("All done!")

[dgmorrisjr]

Thanks @imgroot1391! I like it!

[eksvu]

Good day!

It turns out that many requests to the api will be generated. Maybe there is a method how from all_cve = vulners_api.archive ("cve") (Download whole database collection and work with data locally) to import all data to the operating system