MONAI Deploy and torch models

[cpicarra]

Hello,

Is there any way to make Monai deploy accept torch model weights files (.pt), instead of TorchScript files? The conversion of some complex models to torchscript seems to not be possible, but also haven't been able to make it work with a .pt file (in something similar to this example.

Thank you!

[MMelQin]

@cpicarra Thanks for the question, and this surely can be considered as a feature request.

[ty010624]

Please can you consider this as a feature request?

I found many MONAI bundles in MONAI Model Zoo only provide PyTorch .pt but not TorchScript .ts file, such as swin_unetr_btcv_segmentation, breast_density_classification, etc.

How should we deploy these bundles using Monai Deploy?

[ArpanGyawali]

Yes, I have the same problem. I have trained a model in monai label and it only provides me Pytorch.pt file but not TorchScript.ts file, so i am not been able to deploy it.

Has this feature been added or is there any other solution?
@MMelQin

[MMelQin]

PyTorch has good documentation on TorchScript and sample code to trace/script to a TorchScript model, which has performance advantages in inference use case.

[MingChin-Kao]

Hi~

Recently, I've also been experimenting with training models in MONAI Label and deploying them using the MONAI Deploy App SDK. Like others, I encountered issues with the conversion process (from .pt to .ts).

I used the 'spleen_ct_segmentation' Bundle. After the training and labeling process, I tried converting the model into a .ts file using the method provided by the official PyTorch website. However, I encountered errors related to the inability to retrieve 'bundle_config' during the deployment phase when trying to run the app locally (I was using version 0.5.1 of the MONAI Deploy SDK). Interestingly, these issues didn't arise when using the model.ts downloaded from the official tutorial. This suggests that the problem might lie in the conversion process.

While examining the code, I noticed that the MONAIBundleInferenceOperator extracts configuration information from the model.ts file. The .ts file converted using the PyTorch official site did not contain this configuration information. Later, I came across the monai.bundle.ckpt_export API in MONAI Bundle, which can output a .ts file containing both metadata and configuration. After converting my model with this API and integrating it with the deploy-sdk, it worked successfully.

I didn't see any mention in the MONAI Deploy's tutorial documentation about using this API for model conversion. I'm not entirely sure if this approach might lead to any issues, but for now, it seems to work.