kimuraframework icon indicating copy to clipboard operation
kimuraframework copied to clipboard

How to create empty JSON when no records where scrapped?

Open amitpatelx opened this issue 4 years ago • 1 comments

I am crawling a property site and when there are no properties on the site, the existing json on the disk is not being overwritten.

How to ensure that JSON is written irrespective of records found or not.

amitpatelx avatar Jul 14 '21 09:07 amitpatelx

The workaround is to create empty JSON before crawling open_spider callback

# Empty file before start crawling
def self.open_spider
   File.open('path/to/output.json', 'w') { |file| file.write([].to_s) }
end

amitpatelx avatar Aug 06 '21 07:08 amitpatelx