Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serve] Java Serve improvement #42

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Conversation

liuyang-my
Copy link

No description provided.

Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great step forward. A few high-level comments:

  • As I indicated below, we are moving away from the "InputNode" and "DAGDriver" concepts (they're unnecessarily complicated). For Java, let's prioritize matching the .bind() API for multiple deployments and using the handles that are substituted in the constructor.
  • The recommended way for users to deploy in production these days is via config file (e.g., embedded in a KubeRay CR). Can we include in the proposal adding support to deploy Java this way as well?
  • [P1] As part of this plan, it would be great to have a path forward to unify the handle implementations (e.g., shared C++ core).

reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
@edoakes
Copy link
Contributor

edoakes commented Sep 22, 2023

@liuyang-my looks great now, just need to add the config section!

reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
reps/2023-08-18-serve-java-dag-api.md Outdated Show resolved Hide resolved
chuhan.ly added 6 commits September 28, 2023 19:30
Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
…mentResponse demo. 3. Adjusting the way Python deploys deployment for other languages.

Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
grpc_servicer_functions: []

applications:
- name: app
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this can take args: parameter as well (will be passed in the same way that it is via command line)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, the example of adding args in the config file has been supplemented.

Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Can't wait to see this implemented.

edoakes pushed a commit to ray-project/ray that referenced this pull request Oct 10, 2023
This is about Serve Java Improvement. The goal of this PR is to make the Java API consistent with Python.

The design of the user API is discussed in this proposal: ray-project/enhancements#42.
This PR only covers the Java programming API part, including Deployment.bind, Serve.run, DeploymentHandle, and so on, ensuring that these APIs can be used properly. The alignment of some internal core logic, documentation enhancements, and support for the config file will be submitted in subsequent PRs.

Signed-off-by: chuhan.ly <[email protected]>
Co-authored-by: chuhan.ly <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants