DEBUG: Checking if TestTask2() is complete DEBUG: Checking if TestTask1(param=0) is complete DEBUG: Checking if TestTask1(param=1) is complete DEBUG: Checking if TestTask1(param=2) is complete DEBUG: Checking if TestTask1(param=3) is complete DEBUG: Checking if TestTask1(param=4) is complete DEBUG: Checking if TestTask1(param=5) is complete DEBUG: Checking if TestTask1(param=6) is complete DEBUG: Checking if TestTask1(param=7) is complete DEBUG: Checking if TestTask1(param=8) is complete DEBUG: Checking if TestTask1(param=9) is complete INFO: Informed scheduler that task TestTask2__99914b932b has status PENDING INFO: Informed scheduler that task TestTask1_9_42cdc61e60 has status PENDING INFO: Informed scheduler that task TestTask1_8_fd6efb969d has status PENDING INFO: Informed scheduler that task TestTask1_7_7e79d717a0 has status PENDING INFO: Informed scheduler that task TestTask1_6_8e134626ff has status PENDING INFO: Informed scheduler that task TestTask1_5_f58565226b has status PENDING INFO: Informed scheduler that task TestTask1_4_b2c7d694ab has status PENDING INFO: Informed scheduler that task TestTask1_3_b4b661a1a8 has status PENDING INFO: Informed scheduler that task TestTask1_2_501eaee7af has status PENDING INFO: Informed scheduler that task TestTask1_1_4bd148c8fe has status PENDING INFO: Informed scheduler that task TestTask1_0_2f3347f2bd has status PENDING INFO: Done scheduling tasks INFO: Running Worker with 4 processes DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 11 DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 10 INFO: [pid 20307] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=7) DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 9 INFO: [pid 20308] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=2) DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 8 INFO: [pid 20309] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=1) DEBUG: 4 running tasks, waiting for next task to finish INFO: [pid 20310] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=3) ERROR: [pid 20310] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) failed TestTask1(param=3) Traceback (most recent call last): File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 191, in run new_deps = self._run_get_new_deps() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 129, in _run_get_new_deps task_gen = self.task.run() File "/work/s143369/luigi_bug/test_luigi_bug.py", line 18, in run with self.output().temporary_path() as temp_output_path: File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 301, in temporary_path return _Manager() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 290, in __init__ self.target.fs.mkdir(tmp_dir, parents=True, raise_if_exists=False) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/local_target.py", line 74, in mkdir os.makedirs(path) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists: 'data/test_data/task1' ERROR: [pid 20308] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) failed TestTask1(param=2) Traceback (most recent call last): File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 191, in run new_deps = self._run_get_new_deps() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 129, in _run_get_new_deps task_gen = self.task.run() File "/work/s143369/luigi_bug/test_luigi_bug.py", line 18, in run with self.output().temporary_path() as temp_output_path: File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 301, in temporary_path return _Manager() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 290, in __init__ self.target.fs.mkdir(tmp_dir, parents=True, raise_if_exists=False) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/local_target.py", line 74, in mkdir os.makedirs(path) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists: 'data/test_data/task1' ERROR: [pid 20309] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) failed TestTask1(param=1) Traceback (most recent call last): File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 191, in run new_deps = self._run_get_new_deps() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/worker.py", line 129, in _run_get_new_deps task_gen = self.task.run() File "/work/s143369/luigi_bug/test_luigi_bug.py", line 18, in run with self.output().temporary_path() as temp_output_path: File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 301, in temporary_path return _Manager() File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py", line 290, in __init__ self.target.fs.mkdir(tmp_dir, parents=True, raise_if_exists=False) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/local_target.py", line 74, in mkdir os.makedirs(path) File "/gpfs/work/s143369/.pyenv/versions/3.6.0/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists: 'data/test_data/task1' /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20307] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=7) INFO: Informed scheduler that task TestTask1_3_b4b661a1a8 has status FAILED DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 8 DEBUG: 4 running tasks, waiting for next task to finish INFO: [pid 20320] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=5) INFO: Informed scheduler that task TestTask1_2_501eaee7af has status FAILED DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 8 /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20320] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=5) DEBUG: 4 running tasks, waiting for next task to finish INFO: [pid 20322] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=8) INFO: Informed scheduler that task TestTask1_1_4bd148c8fe has status FAILED DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 8 /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20322] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=8) DEBUG: 4 running tasks, waiting for next task to finish INFO: Informed scheduler that task TestTask1_7_7e79d717a0 has status DONE DEBUG: Asking scheduler for work... INFO: [pid 20325] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=9) DEBUG: Pending tasks: 7 DEBUG: 4 running tasks, waiting for next task to finish INFO: Informed scheduler that task TestTask1_5_f58565226b has status DONE DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 6 INFO: [pid 20327] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=4) DEBUG: 4 running tasks, waiting for next task to finish INFO: Informed scheduler that task TestTask1_8_fd6efb969d has status DONE DEBUG: Asking scheduler for work... INFO: [pid 20329] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) running TestTask1(param=0) DEBUG: Pending tasks: 5 DEBUG: 4 running tasks, waiting for next task to finish /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20327] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=4) INFO: Informed scheduler that task TestTask1_9_42cdc61e60 has status DONE DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: TestTask1_4_b2c7d694ab is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) DEBUG: TestTask1_0_2f3347f2bd is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) DEBUG: TestTask1_6_8e134626ff is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) INFO: Informed scheduler that task TestTask1_4_b2c7d694ab has status DONE DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: TestTask1_0_2f3347f2bd is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) DEBUG: TestTask1_6_8e134626ff is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20329] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=0) INFO: Informed scheduler that task TestTask1_0_2f3347f2bd has status DONE DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: TestTask1_6_8e134626ff is currently run by worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) /gpfs/work/s143369/.pyenv/versions/3.6.0/envs/luigi-test/lib/python3.6/site-packages/luigi/target.py:171: UserWarning: File system LocalFileSystem client doesn't support atomic mv. warnings.warn("File system {} client doesn't support atomic mv.".format(self.__class__.__name__)) INFO: [pid 20333] Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) done TestTask1(param=6) INFO: Informed scheduler that task TestTask1_6_8e134626ff has status DONE DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: There are 4 pending tasks possibly being run by other workers DEBUG: There are 4 pending tasks unique to this worker DEBUG: There are 4 pending tasks last scheduled by this worker INFO: Worker Worker(salt=986528271, workers=4, host=wdev0, username=s143369, pid=20269) was stopped. Shutting down Keep-Alive thread INFO: ===== Luigi Execution Summary ===== Scheduled 11 tasks of which: * 7 ran successfully: - 7 TestTask1(param=0,4,5,6,7,8,9) * 3 failed: - 3 TestTask1(param=1,2,3) * 1 were left pending, among these: * 1 had failed dependencies: - 1 TestTask2() This progress looks :( because there were failed tasks ===== Luigi Execution Summary =====