Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualization: Compare best,median,worse cases for all benchmarked methods #21

Open
msrepo opened this issue May 18, 2023 · 7 comments
Open
Assignees

Comments

@msrepo
Copy link
Collaborator

msrepo commented May 18, 2023

  • bring in code from MeshVisualizer that uses fury and xvfb
    try thin instead of vedo because of aesthetics
  • use filter_runs_from_wandb to find run-id for each anatomy and method under consideration
  • find evaluation/metrics.csv and find the best, median and worse case sample-id
@msrepo msrepo self-assigned this May 18, 2023
@msrepo
Copy link
Collaborator Author

msrepo commented May 19, 2023

pandas has a weird API

print(df.nlargest(1, ["DSC"], "first")["subject-id"].values[0])
print(df.nsmallest(1, ["DSC"], "first")["subject-id"].values[0])
print(df[df.DSC == df.median(numeric_only=True)["DSC"]]["subject-id"].values[0])

@msrepo
Copy link
Collaborator Author

msrepo commented May 20, 2023

Median Results
median_coronal
median_coronal
median_coronal
median_coronal

@msrepo
Copy link
Collaborator Author

msrepo commented May 21, 2023

pick random samples instead of median, worse, best.
currently, the results are stored as follows:
image
keep the same configuration, but we need a way to identify random samples for tiling later.

def save_montage(ANATOMY, subject_type):

will also need to be modified. either track the subject-id or find a way to consistently track the random samples by consistent naming convention.

@msrepo
Copy link
Collaborator Author

msrepo commented May 21, 2023

Useful comparative visualization
image

@msrepo
Copy link
Collaborator Author

msrepo commented May 21, 2023

Median Results median_coronal median_coronal median_coronal median_coronal

The prediction resolution resampling and metadata issue ,see #1, has risen its ugly head here. I must have changed the parameters when evaluating attentionunet results for vertebra.

@msrepo
Copy link
Collaborator Author

msrepo commented May 21, 2023

Median Results median_coronal median_coronal median_coronal median_coronal

The prediction resolution resampling and metadata issue ,see #1, has risen its ugly head here. I must have changed the parameters when evaluating attentionunet results for vertebra.

Without resampling, metadata copied from groundtruth
image
without resampling, without metadata copied from groundtruth
image

both of these are wrong.

  1. either set resample=True when you copy metada from groundtruth
    image
    The preview happened correctly even if the data and metadata did not match in this case because we reslice when previewing.
    image

  2. or set resample=False and set the correct metadata manually.

@msrepo
Copy link
Collaborator Author

msrepo commented Jun 1, 2023

fixed visualization issues: see #28
before and after
median_coronal

median_coronal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant