Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nightly load test results #2235

Closed
Hugoch opened this issue Jul 15, 2024 · 4 comments
Closed

Nightly load test results #2235

Hugoch opened this issue Jul 15, 2024 · 4 comments

Comments

@Hugoch
Copy link
Member

Hugoch commented Jul 15, 2024

This issue collects load test results runs on nightly builds.
@github-actions GitHub Actions
Copy link

🚀 Load test results are in for commit bee5616cfc2a93e721bfc06a58bebf13f85cae45

Variable length prompts

Constant length prompts

Delta to last release

Metric Input Type Test Type Avg Delta (%)
requests_ok sharegpt_conversations constant_vus 0
error_rate sharegpt_conversations constant_vus 0
end_to_end_latency sharegpt_conversations constant_vus 0
inter_token_latency sharegpt_conversations constant_vus 0
time_to_first_token sharegpt_conversations constant_vus 0
tokens_throughput sharegpt_conversations constant_vus 0
requests_ok sharegpt_conversations constant_arrival_rate 0
error_rate sharegpt_conversations constant_arrival_rate 0
end_to_end_latency sharegpt_conversations constant_arrival_rate 0
inter_token_latency sharegpt_conversations constant_arrival_rate 0
time_to_first_token sharegpt_conversations constant_arrival_rate 0
tokens_throughput sharegpt_conversations constant_arrival_rate 0
requests_ok constant_tokens constant_vus 0
error_rate constant_tokens constant_vus 0
end_to_end_latency constant_tokens constant_vus 0
inter_token_latency constant_tokens constant_vus 0
time_to_first_token constant_tokens constant_vus 0
tokens_throughput constant_tokens constant_vus 0
requests_ok constant_tokens constant_arrival_rate 0
error_rate constant_tokens constant_arrival_rate 0
end_to_end_latency constant_tokens constant_arrival_rate 0
inter_token_latency constant_tokens constant_arrival_rate 0
time_to_first_token constant_tokens constant_arrival_rate 0
tokens_throughput constant_tokens constant_arrival_rate 0

@huggingface huggingface deleted a comment from github-actions bot Jul 16, 2024
@huggingface huggingface deleted a comment from github-actions bot Jul 16, 2024
@Hugoch Hugoch pinned this issue Jul 16, 2024
@Hugoch Hugoch mentioned this issue Jul 16, 2024
Closed
5 tasks
@github-actions GitHub Actions
Copy link

🚀 Load test results are in for commit ceac61a0267331875b355fd439c091c77fff139f

Variable length prompts

Constant length prompts

Delta to last release

Metric Input Type Test Type Avg Delta (%)
requests_ok sharegpt_conversations constant_vus 0
error_rate sharegpt_conversations constant_vus 0
end_to_end_latency sharegpt_conversations constant_vus 0
inter_token_latency sharegpt_conversations constant_vus 0
time_to_first_token sharegpt_conversations constant_vus 0
tokens_throughput sharegpt_conversations constant_vus 0
requests_ok sharegpt_conversations constant_arrival_rate 0
error_rate sharegpt_conversations constant_arrival_rate 0
end_to_end_latency sharegpt_conversations constant_arrival_rate 0
inter_token_latency sharegpt_conversations constant_arrival_rate 0
time_to_first_token sharegpt_conversations constant_arrival_rate 0
tokens_throughput sharegpt_conversations constant_arrival_rate 0
requests_ok constant_tokens constant_vus 0
error_rate constant_tokens constant_vus 0
end_to_end_latency constant_tokens constant_vus 0
inter_token_latency constant_tokens constant_vus 0
time_to_first_token constant_tokens constant_vus 0
tokens_throughput constant_tokens constant_vus 0
requests_ok constant_tokens constant_arrival_rate 0
error_rate constant_tokens constant_arrival_rate 0
end_to_end_latency constant_tokens constant_arrival_rate 0
inter_token_latency constant_tokens constant_arrival_rate 0
time_to_first_token constant_tokens constant_arrival_rate 0
tokens_throughput constant_tokens constant_arrival_rate 0

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024

🚀 Load test results are in for commit 893bd3c924c341a3b7a1c53a5a6bd3379501bd2d

Variable length prompts

Constant length prompts

Delta to last release

Metric Input Type Test Type Avg Delta (%)
requests_ok sharegpt_conversations constant_vus 0
error_rate sharegpt_conversations constant_vus 0
end_to_end_latency sharegpt_conversations constant_vus 0
inter_token_latency sharegpt_conversations constant_vus 0
time_to_first_token sharegpt_conversations constant_vus 0
tokens_throughput sharegpt_conversations constant_vus 0
requests_ok sharegpt_conversations constant_arrival_rate 0
error_rate sharegpt_conversations constant_arrival_rate 0
end_to_end_latency sharegpt_conversations constant_arrival_rate 0
inter_token_latency sharegpt_conversations constant_arrival_rate 0
time_to_first_token sharegpt_conversations constant_arrival_rate 0
tokens_throughput sharegpt_conversations constant_arrival_rate 0
requests_ok constant_tokens constant_vus 0
error_rate constant_tokens constant_vus 0
end_to_end_latency constant_tokens constant_vus 0
inter_token_latency constant_tokens constant_vus 0
time_to_first_token constant_tokens constant_vus 0
tokens_throughput constant_tokens constant_vus 0
requests_ok constant_tokens constant_arrival_rate 0
error_rate constant_tokens constant_arrival_rate 0
end_to_end_latency constant_tokens constant_arrival_rate 0
inter_token_latency constant_tokens constant_arrival_rate 0
time_to_first_token constant_tokens constant_arrival_rate 0
tokens_throughput constant_tokens constant_arrival_rate 0

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 2, 2024

🚀 Load test results are in for commit 4e7c0c63c6d2a671ff2306a84212aeb339f1e92b

Variable length prompts

Constant length prompts

Delta to last release

Metric Input Type Test Type Avg Delta (%)
requests_ok sharegpt_conversations constant_vus 0
error_rate sharegpt_conversations constant_vus 0
end_to_end_latency sharegpt_conversations constant_vus 0
inter_token_latency sharegpt_conversations constant_vus 0
time_to_first_token sharegpt_conversations constant_vus 0
tokens_throughput sharegpt_conversations constant_vus 0
requests_ok sharegpt_conversations constant_arrival_rate 0
error_rate sharegpt_conversations constant_arrival_rate 0
end_to_end_latency sharegpt_conversations constant_arrival_rate 0
inter_token_latency sharegpt_conversations constant_arrival_rate 0
time_to_first_token sharegpt_conversations constant_arrival_rate 0
tokens_throughput sharegpt_conversations constant_arrival_rate 0
requests_ok constant_tokens constant_vus 0
error_rate constant_tokens constant_vus 0
end_to_end_latency constant_tokens constant_vus 0
inter_token_latency constant_tokens constant_vus 0
time_to_first_token constant_tokens constant_vus 0
tokens_throughput constant_tokens constant_vus 0
requests_ok constant_tokens constant_arrival_rate 0
error_rate constant_tokens constant_arrival_rate 0
end_to_end_latency constant_tokens constant_arrival_rate 0
inter_token_latency constant_tokens constant_arrival_rate 0
time_to_first_token constant_tokens constant_arrival_rate 0
tokens_throughput constant_tokens constant_arrival_rate 0

@Hugoch Hugoch closed this as completed Oct 3, 2024
@Hugoch Hugoch unpinned this issue Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Hugoch