Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speech interruptions with Kokoro 0.19 on iOS #1796

Open
jankammerath opened this issue Feb 6, 2025 · 0 comments
Open

Speech interruptions with Kokoro 0.19 on iOS #1796

jankammerath opened this issue Feb 6, 2025 · 0 comments

Comments

@jankammerath
Copy link

jankammerath commented Feb 6, 2025

OS: iOS 18.2, 18.3
Device: iPhone 14, iPad 7th Gen, iPhone simulators

The following generated audio outlines the issue, specifically noticable at 03m 04s:
SoundCloud: sherpa-onnx (iOS) sample

This isolated single generated voice output also shows an "f" being cut off at 0:13:
SoundCloud: sherpa-onnx (iOS) isolated sample

The same sample as above, but generated on HuggingFace with ZeroGPU:
SoundCloud: ZeroGPU sample

The implementation is as follows.

/* init the kokoro model for tts */
self.kokoro = sherpaOnnxOfflineTtsKokoroModelConfig(
    model: model.path(),
    voices: voices.path(),
    tokens: tokens.path(),
    dataDir: dataDir
)

/* setup the serpa onnx offline tts wrapper */
let modelConfig = sherpaOnnxOfflineTtsModelConfig(kokoro: self.kokoro!, debug: 0)
var ttsConfig = sherpaOnnxOfflineTtsConfig(model: modelConfig)
self.tts = SherpaOnnxOfflineTtsWrapper(config: &ttsConfig)

/* generate the audio file in the temp folder */
let text = "Insane in the membrane"
let speed: Float = 1.0
let audioFileName = "/tmp/blabla.wav"
let audio = self.tts?.generate(text: text, sid: Int(voiceId), speed: speed)
_ = audio?.save(filename: tempAudioFile.path())

Is there anything wrong with my implementation or is this just the model? I tried Kokoro 0.19 online at Huggingface and couldn't identify a similar issue. Therefore I am not sure if this is just the model, my code or a bug. Hence, reporting it here for potential investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant