Skip to content

Commit

Permalink
clone batch_coords tensor to prevent Too many open files error (#182)
Browse files Browse the repository at this point in the history
When running wsinfer in a windows subsystem for linux environment, i got the error below. Appending a tensor from the dataloader to a list can cause this issue, and cloning the tensor solves it. This commit clones the tensor.

```
RuntimeError: Too many open files. Communication with the workers is no longer possible. Please increase the limit using `ulimit -n` in the shell or change the sharing strategy by calling `torch.multiprocessing.set_sharing_strategy('file_system')` at the beginning of your code
```
  • Loading branch information
kaczmarj authored Aug 12, 2023
1 parent 773de20 commit 977692e
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion wsinfer/modellib/run_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,10 @@ def run_inference(
probs = torch.nn.functional.softmax(logits, dim=1)
else:
probs = torch.sigmoid(logits.squeeze(1))
slide_coords.append(batch_coords.numpy())
# Cloning the tensor prevents memory accumulation and prevents
# the error "RuntimeError: Too many open files". Jakub ran into this
# error when running wsinfer on a slide in Windows Subsystem for Linux.
slide_coords.append(batch_coords.clone().numpy())
slide_probs.append(probs.numpy())

slide_coords_arr = np.concatenate(slide_coords, axis=0)
Expand Down

0 comments on commit 977692e

Please sign in to comment.