fileconveyor icon indicating copy to clipboard operation
fileconveyor copied to clipboard

Can't seem to get arbitrator.py to run with S3

Open paperscissors opened this issue 13 years ago • 6 comments

I'm getting a pretty nondescript error when I try to run arbitrator.py, aside from some deprecated warnings. Not sure if it's something I'm doing with my config.xml or there's some bug I don't know about. Here's the errors:

/sbin/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated from sets import Set, ImmutableSet 2012-02-26 21:46:19,619 - Arbitrator - WARNING - File Conveyor is initializing. 2012-02-26 21:46:19,623 - Arbitrator - WARNING - Loaded config file. /usr/lib/python2.6/site-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

Here's my (slightly modified) config.xml:

(uploaded to cloudapp, this strips out XML tags: http://cl.ly/2Y2d2D2r3s2M1z3H2g3N

Any ideas? Thanks.

paperscissors avatar Feb 26 '12 21:02 paperscissors

it looks like this is a combination of a malformed config.xml and a python depreciation error. i can't make too much of this error message, other than it is tanking on creating the transporter.

2012-03-15 22:36:13,773 - Arbitrator - WARNING - File Conveyor is initializing. 2012-03-15 22:36:13,777 - Arbitrator - WARNING - Loaded config file. Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

paperscissors avatar Mar 15 '12 22:03 paperscissors

Fixed in https://github.com/mnestor/fileconveyor

mnestor avatar Mar 26 '12 23:03 mnestor

thanks, i knew i wasn't crazy.

paperscissors avatar Mar 26 '12 23:03 paperscissors

I'm actually still seeing errors, but most are different, particularly the SECRET_KEY one. not sure if this refers to the "secret_access_key" value in the S3 server transporter group, but otherwise there's no secret_key value on config.xml that I know of.

python arbitrator.py /opt/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated from sets import Set, ImmutableSet 2012-03-27 02:51:17,326 - Arbitrator - WARNING - File Conveyor is initializing. 2012-03-27 02:51:17,329 - Arbitrator - WARNING - Loaded config file. /usr/lib/python2.6/site-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO /usr/lib/python2.6/site-packages/django/conf/init.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty. warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning) Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

paperscissors avatar Mar 27 '12 02:03 paperscissors

This is exactly what I see. I fixed the django SECRET_KEY error by editing my django global settings file (this is confusing but it is different than the SECRET_ACCESS_KEY required by S3). I'm still getting a slew of errors, however:

/usr/local/lib/python2.6/dist-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

mrcwinn avatar Apr 05 '12 22:04 mrcwinn

Looks like ConnectionError is not included in the file from transporters.transporter

chrisivens avatar Apr 30 '12 10:04 chrisivens