add JNI library

[Acs176]

Merge Request: JNI Library for Android Integration

Overview

This merge request introduces a JNI library that enables WhisperKit’s CLI to integrate with Android environments.

Key Contributions

1. JNI Library for Android Integration

• Developed a JNI library that facilitates communication between WhisperKit’s CLI and Android environments.

2. Deployment and Testing Scripts

• run_on_android.sh:
  A new script simplifies running the CLI on an Android device via the ADB shell.
• adb-push.sh Updates:
  The script has been updated to also push:
  • A test audio file (for verifying audio input and transcription).
  • A CLI binary built for the JNI library.
• Build Script Enhancements:
  The build script now accepts an additional jni flag, which was added due to the need to embed the SDL3 library into the .so file. This was necessary to resolve issues with loading shared libraries in the JNI environment.

Alignment with Project Goals

• JNI Support for Android Environments:
  This merge request provides the essential JNI infrastructure needed for WhisperKit’s Android integration.
• Ease of Use:
  The new scripts and build enhancements streamline the workflow for testing and deploying WhisperKit’s JNI library on Android devices.

[bpkeene]

Awesome work, thank you for your PR! Review is in progress, at a glance it looks very nice!

[bpkeene]

As a quick task, can you rebase this change to top-of-tree on main branch and then run bash test/test_build_all.sh from the root project directory? This should be invoked from within the Docker environment.

I believe some of the changes here for android may be incompatible with the linux target.

[bpkeene]

Let's revert this, headers from android NDK should not be used for linux target

[Acs176]

Hi @bpkeene!
First of all, sorry for the delay. I did run into many issues with the rebase and build process so I ended up purging a lot of the code I initially had. You can see now the PR is much cleaner and everything should work as intended.

[Acs176]

I have made sure the compatibility with Linux is kept as it was before.

Regarding the android app, I will add that code later this week, today I managed to run the model with the QNN delegate for the first time, so great news! There are still some issues with the android app but I'm hoping we will manage to solve them with some collaboration :)

Best regards!

[bpkeene]

could we use a unique_ptr? Looking to keep C-style memory management in the C API only (WhisperKit.h, WhisperKit.cpp). Otherwise removing any c-style allocations elsewhere, including internal files

[Acs176]

Hi @bpkeene!
First of all, thank you for taking the time to review my code.

I have added a commit having the WhisperKitConfig as a reference. This allows changes in the config after the runner object was created to take effect, which wasn't possible before.
I have also included the all_msgs bugfix that I mentioned in an issue earlier today. Let me know what you think :)

I pushed the app into one of the branches in my fork. I didn't do a PR yet, maybe it's better to merge the changes separately since it's a lot of code?

Thanks for your input,
Best regards!

[bpkeene]

Awesome work, thank you for making that change! I'll pull this locally and run through some quick tests, otherwise it LGTM! Really appreciate your contribution :) Excellent work!

I pushed the app into one of the branches in my fork. I didn't do a PR yet, maybe it's better to merge the changes separately since it's a lot of code?

Yes, happy to review this separately

[Acs176]

I updated this PR to mention only the JNI lib and opened a separate PR for the android app

[bpkeene]

Awesome work- thank you for your contribution and patience :)