webarchive-indexing icon indicating copy to clipboard operation
webarchive-indexing copied to clipboard

Building a local cluster fails with ValueError

Open ChillarAnand opened this issue 5 years ago • 0 comments

python build_local_zipnum.py zipnum  ~/Downloads/*.cdx.gz

Traceback (most recent call last):
  File "build_local_zipnum.py", line 102, in <module>
    main()
  File "build_local_zipnum.py", line 98, in main
    run_job(r.inputs, r.output, r.shards, r.parallel, r.numlines)
  File "build_local_zipnum.py", line 38, in run_job
    job = ZipNumClusterJob(args)
  File "/home/anand/.virtualenvs/py27/local/lib/python2.7/site-packages/mrjob/job.py", line 78, in __init__
    super(MRJob, self).__init__(self.mr_job_script(), args)
  File "/home/anand/.virtualenvs/py27/local/lib/python2.7/site-packages/mrjob/launch.py", line 109, in __init__
    self.configure_options()
  File "/home/anand/projects/sandbox/webarchive-indexing/zipnumclusterjob.py", line 42, in configure_options
    help='Number of lines per gzipped block')
  File "/home/anand/.virtualenvs/py27/local/lib/python2.7/site-packages/mrjob/launch.py", line 431, in add_passthrough_option
    self.add_passthru_arg(*args, **_optparse_kwargs_to_argparse(**kwargs))
  File "/home/anand/.virtualenvs/py27/local/lib/python2.7/site-packages/mrjob/options.py", line 1596, in _optparse_kwargs_to_argparse
    raise ValueError('invalid option type: %r' % kwargs['type'])
ValueError: invalid option type: <type 'int'>

ChillarAnand avatar Oct 18 '18 15:10 ChillarAnand