WIP: End to End test for vision and audio #386
Conversation
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #386 +/- ##
==========================================
+ Coverage 77.78% 78.99% +1.21%
==========================================
Files 37 37
Lines 2804 2804
==========================================
+ Hits 2181 2215 +34
+ Misses 623 589 -34 ☔ View full report in Codecov by Sentry. |
docs/docs/cli_v2.md
Outdated
| @@ -1,19 +1,14 @@ | |||
| # CLI v2 Documentation | |||
There was a problem hiding this comment.
weird, this is autogenerated differently. Try running make precommit again?
There was a problem hiding this comment.
Running make precommit indicates changes in the markdown file. I checked out the version from your main branch now.
|
Looks good to me! |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
This pull request adds end-to-end tests for audio and vision embeddings, addressing issue #378 and improving test coverage for the Infinity server's multimedia embedding capabilities.
- Added
test_torch_audio.pyandtest_torch_vision.pyin/libs/infinity_emb/tests/end_to_end/for comprehensive audio and image embedding tests - Implemented tests for single and multiple inputs, error handling, and edge cases in both audio and vision embedding scenarios
- Updated
conftest.pyto include default models for audio and vision, and expandedENGINE_METHODSto support new test cases - Highlighted current limitation of only supporting URL-formatted inputs for audio and vision embeddings
- Included skipped tests for text-only inputs, demonstrating the current unsupported behavior
3 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings
|
|
||
| pytest.ENGINE_METHODS = ["embed", "image_embed", "classify", "rerank"] | ||
| pytest.ENGINE_METHODS = ["embed", "image_embed", "classify", "rerank", "audio_embed"] |
Started working on adding end to end tests for audio and vision. #378
Most of the test cases cover sending URLs to the endpoints, because based on my observation this is the only supported mode of operation.
I added a test case which shows the behaviour of sending only text named
test_{audio,vision}_single_text_only. Sending text results in a 422 Status response.Looking through the code it seems the pydantic layer only accepts URL formated strings: See here
Even with disabled pydandic checks it seems theres only suport for url or list of urls based on this code