VulDB_Spider
VulDB_Spider copied to clipboard
hi-KK
would you kind to share the CNVD data ?
Hello, I just encounter an view of using the CNVD hole data to build the AI model to give an advice to handle the problem. I clone your code and run it ,but found the new problem ,cause its site use an policy to prevent the spider and I buy the dynamic IP Pool then it works ,but the speed is very very slow。。。SO, would you kind share your data to me ? sincerely。。。
Yes, that is okay. There is indeed an anti-reptile mechanism on the CNVD website. I used to crawl the website data for more than ten days. I originally found that the CNVD official website has some shared xml files that can be imported into the database, but I later It was found that the XML format was not uniform due to the year problem, so I spent more than ten days crawling the data. Now, you can find a zip file in the CNVD directory. This is the data I crawled to. The password is the file name. You can import it directly after unzipping it. If you have any questions, you can reply at any time, I will try to help you. @justein
@hi-KK very thanks your reply ! I have scrapyed the 2020 year‘s data , later I will share the data to this reply with a baidu netdisk url . Thanks again.
好的，谢谢
------------------ 原始邮件 ------------------ 发件人: "KEY0NE"<[email protected]>; 发送时间: 2021年2月22日(星期一) 下午3:43 收件人: "hi-KK/VulDB_Spider"<[email protected]>; 抄送: "张江源"<[email protected]>; "Comment"<[email protected]>; 主题: Re: [hi-KK/VulDB_Spider] would you kind to share the CNVD data ? (#1)
数据都已经分享上传在对应目录里的zip文件，密码为文件名，解压即可
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.
很久没实验楼了
发件人: zuiss @.> 发送时间: 2021年8月13日 0:27 收件人: hi-KK/VulDB_Spider @.> 抄送: expchan @.>; Comment @.> 主题: Re: [hi-KK/VulDB_Spider] would you kind to share the CNVD data ? (#1)
你好，最新的可以分享一下吗
― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/hi-KK/VulDB_Spider/issues/1#issuecomment-897784563, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKE36VXMM4A35GTJ3ZO2YB3T4PY6FANCNFSM4MFLNLDQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.