Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Object store plugin #45

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

yiweizh-memverge
Copy link

We propose object store plugin interfaces to allow Ray to support other object stores that may provide additional features without disruption of the overall ray object management.

@stephanie-wang stephanie-wang self-assigned this Sep 26, 2023
@stephanie-wang
Copy link
Contributor

Will try to take a look this week!

Copy link
Contributor

@stephanie-wang stephanie-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, and thanks for the proposal!

Overall I think the proposal looks reasonable. The main thing that is missing for me is a validating example. Ideally we would introduce one additional plugin to start, since this will let us validate the proposed API. Do you have one in mind that you're able to contribute to open source? Otherwise, we should work together to find something - off the top of my head:

  • a Redis instance that's shared with non-Ray applications
  • RDMA-based object store

It would also be good to see the CXL example fleshed out a bit more. One thing I didn't quite understand for example was how object fetching would work. Presumably we don't want to transfer the whole object via the normal raylet-raylet protocol anymore? But I wasn't sure how the current API proposal would be able to support that.

Also, if I understand correctly, this is only meant to use one plugin at a time, right?

One small note: could you create a new file with the proposal instead of overwriting README.md? Thanks!

README.md Outdated
virtual Status Authenticate(const std::string& user, const std::string& passwd) = 0;
virtual Status Authenticate(const std::string& secret) = 0;
/// The object storage optimized memory copy.
virtual void MemoryCopy(void* dest, const void* src, size_t len) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you say more about how this would get used?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently memcpy() is used for data copy to object data buffer in writing object chunks (ObjectBufferPool::WriteChunk)
For objects with data buffer on CXL shared memory between hosts, additional optimization may be needed in addition to memcpy() to guarantee the cache coherence so that other hosts can directly access the shared object buffer directly. The plugin can provide that functionality.

README.md Outdated
New virtual functions of `ObjectStoreClientInterface` :
```c++
/// Authentication to the object store.
virtual Status Authenticate(const std::string& user, const std::string& passwd) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be best if this could be part of a more generic startup method, ideally one that already exists.

README.md Outdated

The plugin object store can be specified during Ray startup. The following CLI parameters, or `ray.init()` options are proposed for the plugin.
```c++
plugin_name: Optional[str] = ‘default’
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably best to make this part of the Ray system config, which can be overridden with RAY_<param> env variables.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea.

@leomem
Copy link

leomem commented Nov 1, 2023

Sorry for the delay, and thanks for the proposal!

Overall I think the proposal looks reasonable. The main thing that is missing for me is a validating example. Ideally we would introduce one additional plugin to start, since this will let us validate the proposed API. Do you have one in mind that you're able to contribute to open source? Otherwise, we should work together to find something - off the top of my head:

  • a Redis instance that's shared with non-Ray applications
  • RDMA-based object store

It would also be good to see the CXL example fleshed out a bit more. One thing I didn't quite understand for example was how object fetching would work. Presumably we don't want to transfer the whole object via the normal raylet-raylet protocol anymore? But I wasn't sure how the current API proposal would be able to support that.

Also, if I understand correctly, this is only meant to use one plugin at a time, right?

One small note: could you create a new file with the proposal instead of overwriting README.md? Thanks!

Thank you for the review. It is meant that only one plugin is used. It is assumed that only one type of object store can be used. I'll create a new file as the base for further discussions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants