Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quantized int types #1528

Merged
merged 6 commits into from
Sep 11, 2024
Merged

Add quantized int types #1528

merged 6 commits into from
Sep 11, 2024

Conversation

josevalim
Copy link
Collaborator

EXLA suite is currently failing:

  2) test quantized types s2 (EXLA.BackendTest)
     test/exla/backend_test.exs:202
     ** (RuntimeError) dims and input_strides_in_bytes must have equal sizes, got 1 and 0
     code: tensor = Nx.s2([-2, -1, 1])
     stacktrace:
       (exla 0.8.0) lib/exla/device_buffer.ex:85: EXLA.DeviceBuffer.unwrap!/1
       (exla 0.8.0) lib/exla/device_buffer.ex:47: EXLA.DeviceBuffer.place_on_device/4
       (exla 0.8.0) lib/exla/backend.ex:46: EXLA.Backend.from_binary/3
       test/exla/backend_test.exs:203: (test)

Erlang does not allow us to pass bitstrings to C, only binaries (the number must be divisible by 8). So I am currently padding it, which should be fine, because all XLA cares is about a pointer at the beginning of the data. However, the call still fails. Here is the source that we call into:

https://github.com/openxla/xla/blob/091ef0c6ec9cb48b04d3f764b6c8b549b189d06c/xla/pjrt/pjrt_stream_executor_client.cc#L823

@josevalim josevalim mentioned this pull request Sep 4, 2024
@jonatanklosko
Copy link
Member

For the record openxla/xla#16795.

@josevalim josevalim merged commit 8a9c2b3 into main Sep 11, 2024
7 of 8 checks passed
@josevalim josevalim deleted the jv-int-quant branch September 11, 2024 22:14
@josevalim
Copy link
Collaborator Author

💚 💙 💜 💛 ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants