Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector search feature #1928

Merged
merged 8 commits into from
Jan 16, 2024
Merged

Vector search feature #1928

merged 8 commits into from
Jan 16, 2024

Conversation

snej
Copy link
Collaborator

@snej snej commented Jan 9, 2024

  • API: Added new index type constant and options struct
  • API: Added c4_setExtensionPath function
  • Added new SQLite functions to convert Fleece vectors to raw vector blobs
  • Added vector-query support to QueryParser
  • API: Added new N1QL/SQL++ functions vector_match and vector_distance
  • SQLiteDataFile now loads the vector-search extension if an extension path is registered
  • Fixed sqlite3_log_callback fn to correctly log SQLITE_WARNING code as a warning not an error; also refactored the code to use a switch statement instead of a bunch of ifs.
  • Added ability for tests to assert there were no warnings or errors logged (see Logging.hh)
  • Made a standalone ReadFileByLines function so tests not based on C4Test can call it
  • Xcode: Fixed the iOS tests target so I could test vector-search on an iOS device
  • Removed the SQLITE_OMIT_LOAD_EXTENSION macro since we now use that feature

There is also documentation for the new vector search SQLite extension.

@cbl-bot
Copy link

cbl-bot commented Jan 10, 2024

Code Coverage Results:

Type Percentage
branches 69.92
functions 80.21
instantiations 36.74
lines 80.41
regions 77.02

@snej snej force-pushed the feature/vector-search branch from 346ac7f to e3cef4c Compare January 10, 2024 20:46
@snej snej requested review from borrrden and jianminzhao January 10, 2024 21:11
@snej snej marked this pull request as ready for review January 10, 2024 21:11
@jianminzhao jianminzhao merged commit 1977bc3 into master Jan 16, 2024
@jianminzhao jianminzhao deleted the feature/vector-search branch January 16, 2024 23:19
jianminzhao added a commit that referenced this pull request Jan 17, 2024
The vector search feature (#1928)
callumbirks pushed a commit that referenced this pull request Feb 1, 2024
* Implemented vector search / vector index [API]

* Now supports cosine distance metric [API]

Added type C4VectorMetric and field C4VectorIndexOptions::metric.
Default value should be kC4VectorMetricEuclidean.

* Add faiss to blackduck and correct SQLiteCpp commit

* Updated mobile-vector-search submodule

* Support vector SQ encoding bits and min/max training size [API]

---------

Co-authored-by: Jim Borden <[email protected]>
Co-authored-by: Jianmin Zhao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants