[InferenceSlicer] - it is hard to set specific tile dimensions

[SkalskiP]

Description

I tried to use InferenceSlicer to divide the frame in four equally sized tiles and it turned out to be hard to do.

import numpy as np
import supervision as sv
from inference import get_model

model = get_model(model_id="football-ball-detection-rejhg/3", api_key=ROBOFLOW_API_KEY)

frame_generator = sv.get_video_frames_generator(source_path='/content/2e57b9_0.mp4')
frame = next(frame_generator)

def callback(patch: np.ndarray) -> sv.Detections:
    print(patch.shape)
    result = model.infer(patch, confidence=0.3)[0]
    return sv.Detections.from_inference(result)

slicer = sv.InferenceSlicer(
    callback=callback,
    overlap_filter_strategy=sv.OverlapFilter.NONE,
    slice_wh=(
        (1920 // 2) * 1.1, 
        (1080 // 2) * 1.1
    ),
    overlap_ratio_wh=(0.1, 0.1)
)

detections = slicer(frame).with_nms(threshold=0.1)

I was expecting the code above to produce 4 tiles with 10% overlap, but it created 9. This ended up being very wasteful as InferenceSlicer is expensive to run.

(594, 1056, 3)
(594, 969, 3)
(594, 18, 3)
(545, 1056, 3)
(545, 969, 3)
(545, 18, 3)
(10, 1056, 3)
(10, 969, 3)
(10, 18, 3)

Additional

• Note: Please share a Google Colab with minimal code to test the new feature. We know it's additional work, but it will speed up the review process. The reviewer must test each change. Setting up a local environment to do this is time-consuming. Please ensure that Google Colab can be accessed without any issues (make it public). Thank you! 🙏🏻

[eric220]

@SkalskiP I've created a generate_grid_offset function that takes a grid shape argument (i.e. (2,2) in your case) and returns the required offsets. I can throw together a colab if you're interested.

[SkalskiP]

I'd love to see it! I already started to work on somehow related topics in this PR: #1434

[eric220]

@SkalskiP Here's the colab:
https://colab.research.google.com/drive/1syvRehgUFfu4jt7M8C7KcbSITakI8olt?usp=sharing

It's a rough draft, but you can get the idea. Just set the frame size and grid argument to whatever you want and it slices and dices into even sized tiles.

[SkalskiP]

Colab is private :)

[eric220]

Sorry, not any more. https://colab.research.google.com/drive/1syvRehgUFfu4jt7M8C7KcbSITakI8olt?usp=sharing

…

On Wed, Aug 7, 2024 at 6:26 AM Piotr Skalski ***@***.***> wrote: Colab is private :) — Reply to this email directly, view it on GitHub <#1415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC5VRE547CLJ3ADRTV6WVFTZQH757AVCNFSM6AAAAABLVBBD4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZTGI2DKNJUHE> . You are receiving this because you commented.Message ID: ***@***.***>

[eric220]

@SkalskiP I have circled back to the grid slicer. I recalculated, refactored and brought it into Numpy land. I also added a plotting script for proof of concept. Please check it out and let me know if this is something you think might be nice to add to the InferenceSlicer repo. It solves your issue nicely.
https://colab.research.google.com/drive/1syvRehgUFfu4jt7M8C7KcbSITakI8olt?usp=sharing

[eric220]

So I am pretty sure the correct stride calculation for equisized tiles is:
side_len = 1920
num_divs = 3
overlap_ratio = .15
#STRIDE IS DERIVED FROM
#overlap = strideoverlap_ratio
#side_len = 2(stride-(.5overlap))+(num_divs-2)(stride-overlap)
stride = int((side_len/(2-overlap_ratio+(num_divs-2)(1-overlap_ratio))))
overlap = int(strideoverlap_ratio)