Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for cluster validations in gRPC clients #10788

Draft
wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

Add support for cluster validations in gRPC clients #10788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
0161aef
Ingester client: configure cluster validation label via common config…
duricanikolic Mar 3, 2025
7c46af2
Merge remote-tracking branch 'origin/main' into yuri/grpc-cluster-val…
duricanikolic Mar 4, 2025
75e98b6
Improving common field inheritance
duricanikolic Mar 4, 2025
4f13b8b
Allow cluster validation in query-frontend and query-scheduler client…
duricanikolic Mar 4, 2025
e61b653
Allow cluster validation in query-frontend clients of scheduled proce…
duricanikolic Mar 5, 2025
940a665
Replace -common.cluster-validation-label with -common.client-cluster-…
duricanikolic Mar 5, 2025
12faa5c
Merge remote-tracking branch 'origin/main' into yuri/grpc-cluster-val…
duricanikolic Mar 5, 2025
06ea6f7
Upgrade to latest dskit
duricanikolic Mar 5, 2025
a166e0f
Merge remote-tracking branch 'origin/main' into yuri/grpc-cluster-val…
duricanikolic Mar 6, 2025
8f0b382
Upgrade to the latest dskit
duricanikolic Mar 6, 2025
9bddb85
Configure gRPC clients for block-builder->block-builder-scheduler com…
duricanikolic Mar 6, 2025
ed4205b
Configure gRPC clients for query-frontend->query-scheduler communicat…
duricanikolic Mar 6, 2025
80900b8
Configure gRPC clients for querier->store-gateway communications
duricanikolic Mar 6, 2025
2ca25b2
Upgrade dskit
duricanikolic Mar 6, 2025
fa9d0e4
Configure gRPC clients for query-scheduler->query-frontend communicat…
duricanikolic Mar 6, 2025
7a47cd8
Merge remote-tracking branch 'origin/main' into yuri/grpc-cluster-val…
duricanikolic Mar 7, 2025
841c5e0
Configure gRPC clients for ruler->ruler communications
duricanikolic Mar 7, 2025
5eb94ef
Merge remote-tracking branch 'origin/main' into yuri/grpc-cluster-val…
duricanikolic Mar 8, 2025
1923344
Configure gRPC clients for ruler->ruler-query-frontend communications
duricanikolic Mar 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
214 changes: 212 additions & 2 deletions cmd/mimir/config-descriptor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -726,7 +726,7 @@
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define server's cluster validation label.",
"desc": "Optionally define the server's cluster validation label. When the validation is enabled, this value will be compared with the cluster validation label received through the requests.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "server.cluster-validation.label",
Expand All @@ -752,7 +752,7 @@
},
{
"kind": "field",
"name": "softvalidation",
"name": "soft_validation",
"required": false,
"desc": "When enabled, soft cluster label validation will be executed. Can be enabled only together with server.cluster-validation.grpc.enabled",
"fieldValue": null,
Expand Down Expand Up @@ -1868,6 +1868,27 @@
"fieldFlag": "querier.store-gateway-client.tls-min-version",
"fieldType": "string",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "querier.store-gateway-client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -2400,6 +2421,27 @@
"fieldFlag": "ingester.client.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "ingester.client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -5654,6 +5696,27 @@
"fieldFlag": "querier.frontend-client.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "querier.frontend-client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -5916,6 +5979,27 @@
"fieldFlag": "querier.scheduler-client.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "querier.scheduler-client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -6307,6 +6391,27 @@
"fieldFlag": "query-frontend.grpc-client-config.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "query-frontend.grpc-client-config.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -12791,6 +12896,27 @@
"fieldFlag": "ruler.client.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "ruler.client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -13828,6 +13954,27 @@
"fieldFlag": "ruler.query-frontend.grpc-client-config.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "ruler.query-frontend.grpc-client-config.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -16416,6 +16563,27 @@
"fieldFlag": "alertmanager.alertmanager-client.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "alertmanager.alertmanager-client.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -18321,6 +18489,27 @@
"fieldFlag": "query-scheduler.grpc-client-config.connect-backoff-max-delay",
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "block",
"name": "cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "query-scheduler.grpc-client-config.cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down Expand Up @@ -20293,6 +20482,27 @@
],
"fieldValue": null,
"fieldDefaultValue": null
},
{
"kind": "block",
"name": "client_cluster_validation",
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "label",
"required": false,
"desc": "Optionally define the cluster validation label to be sent together with the requests by the clients.",
"fieldValue": null,
"fieldDefaultValue": "",
"fieldFlag": "common.client-cluster-validation.label",
"fieldType": "string",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
"fieldDefaultValue": null
}
],
"fieldValue": null,
Expand Down
22 changes: 21 additions & 1 deletion cmd/mimir/help-all.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-alertmanager.alertmanager-client.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-alertmanager.alertmanager-client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-alertmanager.alertmanager-client.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-alertmanager.alertmanager-client.connect-backoff-max-delay duration
Expand Down Expand Up @@ -987,6 +989,8 @@ Usage of ./cmd/mimir/mimir:
Maximum number of CPUs that can simultaneously processes WAL replay. If it is set to 0, then each TSDB is replayed with a concurrency equal to the number of CPU cores available on the machine.
-blocks-storage.tsdb.wal-segment-size-bytes int
TSDB WAL segments files max size (bytes). (default 134217728)
-common.client-cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-common.storage.azure.account-key string
Azure storage account key. If unset, Azure managed identities will be used for authentication instead.
-common.storage.azure.account-name string
Expand Down Expand Up @@ -1601,6 +1605,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-ingester.client.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-ingester.client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-ingester.client.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-ingester.client.connect-backoff-max-delay duration
Expand Down Expand Up @@ -2117,6 +2123,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-querier.frontend-client.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-querier.frontend-client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-querier.frontend-client.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-querier.frontend-client.connect-backoff-max-delay duration
Expand Down Expand Up @@ -2225,6 +2233,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-querier.scheduler-client.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-querier.scheduler-client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-querier.scheduler-client.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-querier.scheduler-client.connect-backoff-max-delay duration
Expand Down Expand Up @@ -2263,6 +2273,8 @@ Usage of ./cmd/mimir/mimir:
Override the expected name on the server certificate.
-querier.shuffle-sharding-ingesters-enabled
Fetch in-memory series from the minimum set of required ingesters, selecting only ingesters which may have received series since -querier.query-ingesters-within. If this setting is false or -querier.query-ingesters-within is '0', queriers always query all ingesters (ingesters shuffle sharding on read path is disabled). (default true)
-querier.store-gateway-client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-querier.store-gateway-client.tls-ca-path string
Path to the CA certificates to validate server certificate against. If not set, the host's root CA certificates are used.
-querier.store-gateway-client.tls-cert-path string
Expand Down Expand Up @@ -2307,6 +2319,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-query-frontend.grpc-client-config.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-query-frontend.grpc-client-config.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-query-frontend.grpc-client-config.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-query-frontend.grpc-client-config.connect-backoff-max-delay duration
Expand Down Expand Up @@ -2521,6 +2535,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-query-scheduler.grpc-client-config.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-query-scheduler.grpc-client-config.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-query-scheduler.grpc-client-config.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-query-scheduler.grpc-client-config.connect-backoff-max-delay duration
Expand Down Expand Up @@ -2955,6 +2971,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-ruler.client.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-ruler.client.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-ruler.client.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-ruler.client.connect-backoff-max-delay duration
Expand Down Expand Up @@ -3043,6 +3061,8 @@ Usage of ./cmd/mimir/mimir:
Enable backoff and retry when we hit rate limits.
-ruler.query-frontend.grpc-client-config.backoff-retries int
Number of times to backoff and retry before failing. (default 10)
-ruler.query-frontend.grpc-client-config.cluster-validation.label string
[experimental] Optionally define the cluster validation label to be sent together with the requests by the clients.
-ruler.query-frontend.grpc-client-config.connect-backoff-base-delay duration
Initial backoff delay after first connection failure. Only relevant if ConnectTimeout > 0. (default 1s)
-ruler.query-frontend.grpc-client-config.connect-backoff-max-delay duration
Expand Down Expand Up @@ -3176,7 +3196,7 @@ Usage of ./cmd/mimir/mimir:
-server.cluster-validation.grpc.soft-validation
[experimental] When enabled, soft cluster label validation will be executed. Can be enabled only together with server.cluster-validation.grpc.enabled
-server.cluster-validation.label string
[experimental] Optionally define server's cluster validation label.
[experimental] Optionally define the server's cluster validation label. When the validation is enabled, this value will be compared with the cluster validation label received through the requests.
-server.graceful-shutdown-timeout duration
Timeout for graceful shutdowns (default 30s)
-server.grpc-conn-limit int
Expand Down
Loading
Loading