Skip to content

Commit

Permalink
Merge pull request #232 from unum-cloud/main-dev
Browse files Browse the repository at this point in the history
First NuGet Release
  • Loading branch information
ashvardanian authored Aug 28, 2023
2 parents 05707f0 + e3c7531 commit 70706fa
Show file tree
Hide file tree
Showing 10 changed files with 139 additions and 94 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/prerelease.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ jobs:
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
where cl
cl /EHsc /std:c++14 /O2 /LD /Fe:libusearch_c.dll lib.cpp /I. /I../include/ /I../fp16/include/ /I../robin-map/include/
cl /EHsc /std:c++14 /O2 /LD /Fe:libusearch_c.dll lib.cpp /I. /I../include/ /I../fp16/include/
working-directory: ${{ github.workspace }}/c
shell: cmd

Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}

build_csharp:
name: Build USearch libraries
name: Build NuGet
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
Expand All @@ -294,7 +294,7 @@ jobs:
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
where cl
cl /EHsc /std:c++14 /O2 /LD /Fe:libusearch_c.dll lib.cpp /I. /I../include/ /I../fp16/include/ /I../robin-map/include/
cl /EHsc /std:c++14 /O2 /LD /Fe:libusearch_c.dll lib.cpp /I. /I../include/ /I../fp16/include/
working-directory: ${{ github.workspace }}/c
shell: cmd

Expand All @@ -314,7 +314,7 @@ jobs:
publish_csharp:
name: Publish C#
runs-on: ubuntu-latest
needs: [build_usearch_libs]
needs: [build_csharp]
env:
USEARCH_LIBS: ${{ github.workspace }}/csharp/lib
NUGET_DIR: ${{ github.workspace }}/csharp/packages
Expand Down Expand Up @@ -350,7 +350,7 @@ jobs:
name: Build Docs
runs-on: ubuntu-22.04
if: ${{ always() }}
needs: [publish_python, publish_javascript, publish_rust, publish_java, publish_swift, publish_docker]
needs: [publish_python, publish_javascript, publish_rust, publish_java, publish_swift, publish_docker, publish_csharp]
permissions:
contents: write
steps:
Expand Down
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,8 +283,10 @@ Broader functionality is ported per request.

### USearch + AI = Multi-Modal Semantic Search

[![USearch Semantic Image Search](https://github.com/ashvardanian/usearch-images/raw/main/assets/usearch-images-slow.gif)](https://github.com/ashvardanian/usearch-images)

AI has a growing number of applications, but one of the coolest classic ideas is to use it for Semantic Search.
One can take an encoder model, like the multi-modal UForm, and a web-programming framework, like UCall, and build a text-to-image search platform in just 20 lines of Python.
One can take an encoder model, like the multi-modal [UForm](https://github.com/unum-cloud/uform), and a web-programming framework, like UCall, and build a text-to-image search platform in just 20 lines of Python.

```python
import ucall
Expand Down Expand Up @@ -314,13 +316,14 @@ def search(query: str) -> np.ndarray:
server.run()
```

A more complete [demo with Streamlit is available on GitHub](https://github.com/ashvardanian/usearch-images).
We have pre-processed some commonly used datasets, cleaned the images, produced the vectors, and pre-built the index.

| Dataset | Modalities | Images | Download |
| :------------------------------------- | --------------------: | -----: | ------------------------------------: |
| [Unsplash 25K][unsplash-25k-origin] | Images & Descriptions | 25 K | [HuggingFace / Unum][unsplash-25k-hf] |
| [Conceptual Captions 3M][cc-3m-origin] | Images & Descriptions | 3 M | [HuggingFace / Unum][cc-3m-hf] |
| [Arxiv 2M][arxiv-2m-origin] | Titles & Abstracts | 2 M | [HuggingFace / Unum][arxiv-2m-hf] |
| Dataset | Modalities | Images | Download |
| :---------------------------------- | --------------------: | -----: | ------------------------------------: |
| [Unsplash][unsplash-25k-origin] | Images & Descriptions | 25 K | [HuggingFace / Unum][unsplash-25k-hf] |
| [Conceptual Captions][cc-3m-origin] | Images & Descriptions | 3 M | [HuggingFace / Unum][cc-3m-hf] |
| [Arxiv][arxiv-2m-origin] | Titles & Abstracts | 2 M | [HuggingFace / Unum][arxiv-2m-hf] |

[unsplash-25k-origin]: https://github.com/unsplash/datasets
[cc-3m-origin]: https://huggingface.co/datasets/conceptual_captions
Expand Down Expand Up @@ -364,15 +367,19 @@ matches = index.search(fingerprints, 10)

### USearch + POI Coordinates = GIS Applications... on iOS?

With Objective-C and iOS bindings, USearch can be easily used in mobile applications
[![USearch Maps with SwiftUI](https://github.com/ashvardanian/SwiftVectorSearch/raw/main/USearch+SwiftUI.gif)](https://github.com/ashvardanian/SwiftVectorSearch)

With Objective-C and Swift iOS bindings, USearch can be easily used in mobile applications.
The [SwiftVectorSearch](https://github.com/ashvardanian/SwiftVectorSearch) project illustrates how to build a dynamic, real-time search system on iOS.
In this example, we use 2-dimensional vectors—encoded as latitude and longitude—to find the closest Points of Interest (POIs) on a map.
The search is based on the Haversine distance metric, but can easily be extended to support high-dimensional vectors.

## Integrations

- [x] GPTCache: [Python](https://github.com/zilliztech/GPTCache/releases/tag/0.1.29).
- [x] LangChain: [Python](https://github.com/langchain-ai/langchain/releases/tag/v0.0.257) and [JavaScipt](https://github.com/hwchase17/langchainjs/releases/tag/0.0.125).
- [x] ClickHouse: [C++](https://github.com/ClickHouse/ClickHouse/pull/53447).
- [ ] Microsoft Semantic Kernel: [Python](https://github.com/microsoft/semantic-kernel/pull/2358) and C#.
- [x] Microsoft Semantic Kernel: [Python](https://github.com/microsoft/semantic-kernel/releases/tag/python-0.3.9.dev) and C#.

## Citations

Expand Down
6 changes: 3 additions & 3 deletions c/lib.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ add_result_t add_(index_dense_t* index, usearch_key_t key, void const* vector, s
case scalar_kind_t::f32_k: return index->add(key, (f32_t const*)vector);
case scalar_kind_t::f64_k: return index->add(key, (f64_t const*)vector);
case scalar_kind_t::f16_k: return index->add(key, (f16_t const*)vector);
case scalar_kind_t::i8_k: return index->add(key, (i8_bits_t const*)vector);
case scalar_kind_t::i8_k: return index->add(key, (i8_t const*)vector);
case scalar_kind_t::b1x8_k: return index->add(key, (b1x8_t const*)vector);
default: return add_result_t{}.failed("Unknown scalar kind!");
}
Expand All @@ -59,7 +59,7 @@ std::size_t get_(index_dense_t* index, usearch_key_t key, size_t count, void* ve
case scalar_kind_t::f32_k: return index->get(key, (f32_t*)vector, count);
case scalar_kind_t::f64_k: return index->get(key, (f64_t*)vector, count);
case scalar_kind_t::f16_k: return index->get(key, (f16_t*)vector, count);
case scalar_kind_t::i8_k: return index->get(key, (i8_bits_t*)vector, count);
case scalar_kind_t::i8_k: return index->get(key, (i8_t*)vector, count);
case scalar_kind_t::b1x8_k: return index->get(key, (b1x8_t*)vector, count);
default: return search_result_t().failed("Unknown scalar kind!");
}
Expand All @@ -70,7 +70,7 @@ search_result_t search_(index_dense_t* index, void const* vector, scalar_kind_t
case scalar_kind_t::f32_k: return index->search((f32_t const*)vector, n);
case scalar_kind_t::f64_k: return index->search((f64_t const*)vector, n);
case scalar_kind_t::f16_k: return index->search((f16_t const*)vector, n);
case scalar_kind_t::i8_k: return index->search((i8_bits_t const*)vector, n);
case scalar_kind_t::i8_k: return index->search((i8_t const*)vector, n);
case scalar_kind_t::b1x8_k: return index->search((b1x8_t const*)vector, n);
default: return search_result_t().failed("Unknown scalar kind!");
}
Expand Down
16 changes: 8 additions & 8 deletions include/usearch/index_dense.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -584,31 +584,31 @@ class index_dense_gt {

// clang-format off
add_result_t add(key_t key, b1x8_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_b1x8); }
add_result_t add(key_t key, i8_bits_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_i8); }
add_result_t add(key_t key, i8_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_i8); }
add_result_t add(key_t key, f16_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_f16); }
add_result_t add(key_t key, f32_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_f32); }
add_result_t add(key_t key, f64_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_f64); }

search_result_t search(b1x8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_b1x8); }
search_result_t search(i8_bits_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_i8); }
search_result_t search(i8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_i8); }
search_result_t search(f16_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f16); }
search_result_t search(f32_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f32); }
search_result_t search(f64_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f64); }

std::size_t get(key_t key, b1x8_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_b1x8); }
std::size_t get(key_t key, i8_bits_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_i8); }
std::size_t get(key_t key, i8_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_i8); }
std::size_t get(key_t key, f16_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_f16); }
std::size_t get(key_t key, f32_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_f32); }
std::size_t get(key_t key, f64_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_f64); }

cluster_result_t cluster(b1x8_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_b1x8); }
cluster_result_t cluster(i8_bits_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_i8); }
cluster_result_t cluster(i8_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_i8); }
cluster_result_t cluster(f16_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_f16); }
cluster_result_t cluster(f32_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_f32); }
cluster_result_t cluster(f64_t const* vector, std::size_t level, std::size_t thread = any_thread()) const { return cluster_(vector, level, thread, casts_.from_f64); }

aggregated_distances_t distance_between(key_t key, b1x8_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_b1x8); }
aggregated_distances_t distance_between(key_t key, i8_bits_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_i8); }
aggregated_distances_t distance_between(key_t key, i8_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_i8); }
aggregated_distances_t distance_between(key_t key, f16_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_f16); }
aggregated_distances_t distance_between(key_t key, f32_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_f32); }
aggregated_distances_t distance_between(key_t key, f64_t const* vector, std::size_t thread = any_thread()) const { return distance_between_(key, vector, thread, casts_.to_f64); }
Expand Down Expand Up @@ -1872,13 +1872,13 @@ class index_dense_gt {
casts_t result;

result.from_b1x8 = cast_gt<b1x8_t, to_scalar_at>{};
result.from_i8 = cast_gt<i8_bits_t, to_scalar_at>{};
result.from_i8 = cast_gt<i8_t, to_scalar_at>{};
result.from_f16 = cast_gt<f16_t, to_scalar_at>{};
result.from_f32 = cast_gt<f32_t, to_scalar_at>{};
result.from_f64 = cast_gt<f64_t, to_scalar_at>{};

result.to_b1x8 = cast_gt<to_scalar_at, b1x8_t>{};
result.to_i8 = cast_gt<to_scalar_at, i8_bits_t>{};
result.to_i8 = cast_gt<to_scalar_at, i8_t>{};
result.to_f16 = cast_gt<to_scalar_at, f16_t>{};
result.to_f32 = cast_gt<to_scalar_at, f32_t>{};
result.to_f64 = cast_gt<to_scalar_at, f64_t>{};
Expand All @@ -1891,7 +1891,7 @@ class index_dense_gt {
case scalar_kind_t::f64_k: return make_casts_<f64_t>();
case scalar_kind_t::f32_k: return make_casts_<f32_t>();
case scalar_kind_t::f16_k: return make_casts_<f16_t>();
case scalar_kind_t::i8_k: return make_casts_<i8_bits_t>();
case scalar_kind_t::i8_k: return make_casts_<i8_t>();
case scalar_kind_t::b1x8_k: return make_casts_<b1x8_t>();
default: return {};
}
Expand Down
Loading

0 comments on commit 70706fa

Please sign in to comment.