reset connections

[warmerdam]

I am getting errors like this from time to time. Possibly it would help to do a retry or something?

+ l8_process_scene.py --verbose -s s3queue --clean --list-file job_20719258.csv LC80011152015035LGN00
.....Traceback (most recent call last):
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 110, in <module>
    status = main(sys.argv[1:])
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 101, in main
    overwrite = args.overwrite)
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 56, in process
    verbose=verbose)
  File "/opt/planet/programs/landsat_ingestor/ingestor/puller.py", line 18, in pull
    return puller_s3queue.pull(scene_root, scene_dict, verbose=verbose)
  File "/opt/planet/programs/landsat_ingestor/ingestor/puller_s3queue.py", line 34, in pull
    for d in rv.iter_content(chunk_size=1024 * 1024 * 10):
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 616, in generate
    decode_content=True):
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 236, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 183, in read
    data = self._fp.read(amt)
  File "/usr/lib/python2.7/httplib.py", line 561, in read
    s = self.fp.read(amt)
  File "/usr/lib/python2.7/socket.py", line 380, in read
    data = self._sock.recv(left)
  File "/usr/lib/python2.7/ssl.py", line 241, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 160, in read
    return self._sslobj.read(len)
socket.error: [Errno 104] Connection reset by peer

[warmerdam]

A similar failure that would presumably benefit from appropriate retry.

+ python -u /opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py --verbose -s s3queue --clean --overwrite --list-file job_52562854.csv LC81150352015226LGN00
.......
.....
.......
...................................................................Traceback (most recent call last):
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 115, in <module>
    status = main(sys.argv[1:])
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 106, in main
    overwrite = args.overwrite)
  File "/opt/planet/programs/landsat_ingestor/ingestor/l8_process_scene.py", line 56, in process
    verbose=verbose)
  File "/opt/planet/programs/landsat_ingestor/ingestor/puller.py", line 18, in pull
    return puller_s3queue.pull(scene_root, scene_dict, verbose=verbose)
  File "/opt/planet/programs/landsat_ingestor/ingestor/puller_s3queue.py", line 49, in pull
    for d in rv.iter_content(chunk_size=1024 * 1024 * 10):
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 638, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 256, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 186, in read
    data = self._fp.read(amt)
  File "/usr/lib/python2.7/httplib.py", line 569, in read
    s = self.fp.read(amt)
  File "/usr/lib/python2.7/socket.py", line 380, in read
    data = self._sock.recv(left)
socket.error: [Errno 104] Connection reset by peer
Task ended with status 1

[warmerdam]

@kapadia - feel free to take this if you would like.