RedditDownloader icon indicating copy to clipboard operation
RedditDownloader copied to clipboard

CSV data not importing

Open slormo opened this issue 2 years ago • 0 comments

Only a tiny slither of posts (if any) are imported from the post_votes.csv when importing from the data request

Ran on Windows 10 using the sourcecode of version 3.1.5

command used: python redditdownloader --import_csv=data/post_votes.csv --full_csv post_votes.csv is located at D:\RedditDownloader-3.1.5\data Run.py and \redditdownloader\ are located in D:\RedditDownloader-3.1.5

The log when using the command is as follows:

  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()

  File "D:\RedditDownloader-3.1.5\redditdownloader\processing\redditloader.py", line 30, in run
    self.load()
Remaining: 113/114+
  File "D:\RedditDownloader-3.1.5\redditdownloader\processing\redditloader.py", line 51, in load
    self._scan_sources()

  File "D:\RedditDownloader-3.1.5\redditdownloader\processing\redditloader.py", line 65, in _scan_sources
    for r in source.get_elements():
File:   File "D:\RedditDownloader-3.1.5\redditdownloader\sources\direct_file_source.py", line 39, in get_elements
    for e in self.get_subs(subs):
*downloaded file*  File "D:\RedditDownloader-3.1.5\redditdownloader\sources\direct_file_source.py", line 61, in get_subs
    for post in self._ps.search_submissions(ids=subs, limit=len(subs)):

  Handler:  File "C:\Users\USER\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py", line 238, in _search
    for response in self._handle_paging(url):
   File "C:\Users\USER\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py", line 218, in _handle_paging
    received_size = int(data['metadata']['size'])
ytdl
KeyError: 'metadata'
   Status: Downloading video...
  Percent: 7.0%
Traceback (most recent call last):
File: *downloaded file*
  Handler: ytdl
   Status: Downloading video...
  Percent: 0.7%

File: *downloaded file*
  Handler: ytdl
   Status: Downloading video...
  Percent: 49.9%

File: *downloaded file*
  Handler: ytdl
   Status: Downloading video...
  Percent: 26.3%

File: *downloaded file*
  Handler: ytdl
   Status: Looking for video...

All finished.

slormo avatar Jul 18 '22 23:07 slormo