deepracer-local
deepracer-local copied to clipboard
Cannot connect to Redis when using DEEP_CONVOLUTIONAL_NETWORK
trafficstars
when I try using DEEP_CONVOLUTIONAL_NETWORK (before used DEEP_CONVOLUTIONAL_NETWORK_SHALLOW, It run smoothly).
It raises an error in evaluation step:
Training> Name=main_level/agent, Worker=0, Episode=20, Total reward=2.1, Steps=99, Training iteration=0
## agent: Starting evaluation phase
Testing> Name=main_level/agent, Worker=0, Episode=20, Total reward=1.5, Steps=99, Training iteration=0
## agent: Finished evaluation phase. Success rate = 0.0, Avg Total Reward = 1.5
## agent: Starting evaluation phase
Exception in thread Thread-435:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 185, in _read_from_socket
raise socket.error(SERVER_CLOSED_CONNECTION_ERROR)
OSError: Connection closed by server.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 3236, in run
timeout=sleep_time)
File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 3135, in get_message
response = self.parse_response(block=False, timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 3036, in parse_response
return self._execute(connection, connection.read_response)
File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 3013, in _execute
return command(*args)
File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 637, in read_response
response = self._parser.read_response()
File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 290, in read_response
response = self._buffer.readline()
File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 224, in readline
self._read_from_socket()
File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 199, in _read_from_socket
(e.args,))
redis.exceptions.ConnectionError: Error while reading from socket: ('Connection closed by server.',)
I'm afraid I don't have enough memory on my GPU to test a 5 layer network.
What GPU are you running? Perhaps Redis is being killed due to oom?