Faster image loading on Queue after batch has been transferred to GPU.

[ramonemiliani93]

Hi! I was taking a look at the Queue explanation here. I am not sure if understood correctly, but it seems the queue launches the workers to process the subjects only when empty; please correct me if I am wrong. If this is the case do you think there is a way to launch a worker as soon as a batch is transferred to the GPU instead of waiting until the queue is empty, maybe something like this can help.

[fepegar]

it seems the queue launches the workers to process the subjects only when empty

As far as I know, the loaders keep working in the background all the time. Moreover, if you instantiate the queue with start_background=True (default), they actually start working as soon as the queue is instantiated, before the patches list needs to be filled.

[ramonemiliani93]

@fepegar thanks for the quick reply! Maybe im not completely understanding how it is working under the hood. Just to clarify, on the queue, specifically this part:

    def __getitem__(self, _):
        # There are probably more elegant ways of doing this
        if not self.patches_list:
            self._print('Patches list is empty.')
            self._fill()
        sample_patch = self.patches_list.pop()
        self.num_sampled_patches += 1
        return sample_patch

The queue will be filled only when the patches list is empty. As soon as the last batch is processed all the workers will start filling up the patches list and the GPU might not be processing any batch.

[fepegar]

The queue will be filled only when the patches list is empty.

Correct.

As soon as the last batch is processed all the workers will start filling up the patches list and the GPU might not be processing any batch.

Partially correct, I think. I believe the workers keep working after the queue has been filled, and so those volumes can be sampled quickly when the queue needs to be filled again. However, it's true that the GPU would be idle between the time in which the augmented volumes have been delivered by the workers and the time in which the queue is full, which is bad. This has prompted some efforts to improve this behavior (#311, #393), but unfortunately reviewing and testing these take time and I am currently very busy with my PhD, so can't really work on those big TorchIO changes. To overcome idle GPUs, I suggest maximizing the queue length and the samples per volume, which would increase the RAM usage, though.

[ramonemiliani93]

Thanks a lot 🙌 I'll try your suggestions and take an in depth look at both issues and the suggested improvements. Good luck with your PhD!

[fepegar]

Thank you :)