Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce API to retrieve vshard-topology for storage and router for etcd.cluster.vshard #30

Open
ochaton opened this issue Feb 12, 2024 · 1 comment

Comments

@ochaton
Copy link
Member

ochaton commented Feb 12, 2024

We have several installations with moonlibs/config as Configuration Driver for multiple vshard-clusters. All of them have to copy-paste function build_sharding which retrieves entire configuration under etcd/prefix and constructs vshard acceptable configuration.

After introducing master_selection_policy = etcd.cluster.vshard and having config.etcd.get_all actually we are able to standardize the way every installation should configure it's vshard cluster.

I propose to specify configuration of vshard like that:

common:
  vshard: // main vshard section shared across all instances
    bucket_count: XXX // 
    collect_bucket_garbage_interval: ...
    collect_lua_garbage: ...
    sync_timeout: ...
    rebalancer_disbalance_threshold: ...
    rebalancer_max_sending: ...
    discovery_mode: ...
    consts: // the way to override vshard consts. The moonlibs/config should not override somehow vshard.consts by himself.
        BUCKET_CHUNK_SIZE: 3000
    weights: // the way to specify distances between zones (the higher the longer)
      region-1:
        region-2: 100 // distances between zones, (default is none)
clusters:
  shard_001:
    vshard:
      weight: 1 // weight of replicaset in terms of rebalancing/resharding (default none)
instances:
  instance_001_01:
    vshard:
      zone: "region-1" // replica weight in terms of router API (not the same as replicaset weight)

moonlibs/config should not call vshard.storage.cfg or vshard.router.cfg. It should provide API as config.vshard_get("storage") and config.vshard_get("router") to fetch constructed topology for storage or router instead.

@ochaton
Copy link
Member Author

ochaton commented Feb 12, 2024

Maybe, it is better to name it config:sharding() as it is done in Tarantool 3.0 config

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant