Cluster: K8s: Proxy to extend origin servers.

[winlinvip]

Both cluster and proxy works for system load balancing, in short to serve a large set of connections or clients, but there are some differences.

中文: Cluster和Proxy都是为了解决系统的负载均衡问题，简单来说就是支持很多连接或客户端，但是它们从方案上看是有区别的。

Cluster works as an integrity, a set of servers of a cluster works like one server. So it supports a large number of streams, and each stream supports a lots of connections. For example, a cluster supports 100k streams and 100m viewers, or infinety system capacity.

中文：Cluster作为一个整体工作，很多服务器工作起来像是单个服务器一样。Cluster支持很多流，每个流支持很多播放连接。比如，Cluster支持10万路流，支持1亿播放观看，或者可以简单认为Cluster的系统容量是无限的。

Proxy works as a agent of SRS Origin server, to proxy streams to a special set of origin servers, and proxy to the same server for a specifical stream. Proxy doesn't extend system capacity for it proxy each UDP or TCP packet, but it helps media server for load balancing.

中文：Proxy是作为源站的代理，将流代理到一系列的源站，并且某个特定的流只会代理到同一个源站。Proxy并不会增加系统的容量，因为Proxy转发了每个UDP或TCP包，但是Proxy能帮助实现系统的负载均衡。从系统的整体来看，Proxy也可以实现系统能力的扩容。

Proxy is designed to make origin server stateless, to build a cluster from isolated stateless origin servers to a cluster, and can also be a part of a cluster.

中文：Proxy是为了解决源站的状态问题，将源站变成无状态服务。Proxy可以将多个无关的源站，组建成一个源站集群。因此Proxy当然也是集群的一个部分。

Architecture

Proxy works with SRS Origin servers, the stream flow works like this:

中文：Proxy一般和源站服务器一起工作，它的流图如下所示。

Client ---> LB --> Proxy Servers --> Origin Servers

OBS/FFmpeg --RTMP--> K8s(Service) --Proxy--> SRS(pod A)

Browsers --FLV/HLS/SRT--> K8s(Service) --Proxy--> SRS(pod A)

Browsers --+---HTTP-API--> K8s(Service) --Proxy--> SRS(pod A)
           +---WebRTC----> K8s(Service) --Proxy--> SRS(pod A)

LB is load balancer, such as K8s service, or cloud load balancer. Generally, LB binds public internet IP, for clients to connect to.

中文：LB就是负载均衡，比如K8s Service，或者云服务的负载均衡。一般来说，LB绑定了公网IP，对客户端提供服务。

Proxy is stateless, so you're able to deploy a set of proxy servers, for example, use K8s Deployment to deploy a set of proxies.

中文：Proxy是无状态服务，所以你可以很方便的部署一系列的Proxy服务器。例如，可以用K8s Deployment部署很多Proxy。

When client request stream from proxy server, it find stream from its infrastructure, and proxy to the specifical origin server, so that each stream is served by a specifical backend server. For the first time, proxy randomly pick one origin server for a fresh stream.

中文：当客户端连接到Proxy时，它会从基础设施查询流所在的后端服务器，并且将流代理到这台服务器，这样保证每个流是被同一个后端服务器服务的。第一次Proxy随机选择一个后端服务器。

Service Discovery

Regarding service discovery, it involves two aspects: how the proxy discovers the origin servers and how the proxy obtains stream information from other proxy servers.

中文：关于服务发现，它涉及两个方面：Proxy如何发现Origin Server以及Proxy如何从其他Proxy Server获取流信息。

There are two types of service discovery mechanisms: one that caters to simple scenarios like demos or small clusters, and another that provides stability for large-scale online products.

中文：有两种服务发现机制：一种适用于简单场景，如演示或小型集群；另一种为大规模在线产品提供稳定性。

1. Simple Static Discovery: The origin and proxy server IP addresses are configured in the server configuration file. This allows the proxy server to access the target origin server for API and media streams and fetch stream information from other proxy servers.

中文：Simple Static Discovery: Proxy和Origin Server IP 地址直接配置在配置文件。这使得Proxy可以访问目标Origin Server，并从其他Proxy获取流信息。

Client --> Proxy 0-2 --> Origin 0-2

Proxy X config:
id: proxyX
origin: origin0, origin1, origin2

Origin X config:
proxy: proxy0, proxy1, proxy2

2. Robust Dynamic Discovery: The origin and proxy server utilize HTTP API to register and query from a service manager, which employs Redis for storing server and stream metadata. The entire system can be deployed as a Kubernetes deployment.

中文：Robust Dynamic Discovery: Proxy和Origin Server使用 HTTP API 向Service Manager注册和查询，Service Manager使用 Redis 存储服务器和流元数据。整个系统可以作为 Kubernetes Deployment进行部署。

Client --> Proxy 0-2 --> Origin 0-2
Proxy --> Service Manager --> Redis
Origin --> Service Manager --> Redis

Proxy X config:
id: proxyX
origin: service manager

Origin X config:
id: originX
proxy: service manager

In this architecture, both the proxy and origin servers, as well as the service manager, utilize the same HTTP API for service discovery, and the configuration file shares a consistent format across all components.

中文：在此架构中，Proxy和Origin以及Service Manager都使用相同的HTTP API进行服务发现，配置文件在所有组件之间采用一致的格式。

Proxy of Proxies

Edge is actually another type of proxy, but with origin ip configured, but origin pod IP is variant not fixed. While proxy doesn't need to configure the origin IP because it depends on redis or other service discover mechanism, so proxy can also be used for upstream server for edge. In this situation, proxy is like a K8s service of origin servers for edge servers.

中文：其实Edge也是一种Proxy，不过它需要预先配置源站IP地址，但源站Pod的IP总是变化的，并不是固定的IP。由于Proxy使用Redis或其他服务发现机制，所以Proxy并不需要配置源站IP，此时Proxy可以作为Edge的上游服务器，这样就可以避免Edge依赖源站的IP的问题。这种情况下，Proxy实际上被当做了源站的K8s Service使用，Edge回源到这个固定的Service名字。

Client --RTMP--> Edge ---RTMP--> Proxy --RTMP--> Origin
Edge is (K8s Edge Service --RTMP--> K8s Edge Pods)
Proxy is (K8s Proxy Service --RTMP--> K8s Proxy Pods)
Origin is (K8s Origin Pods)

Note: Please note that both edge and proxy is behind K8s service, not only K8s pods, while Origin is a set of K8s pods.

Note: 请注意Edge和Proxy都是部署在K8s service后面的，而不是直接的Pods。而Origin源站则是直接部署的K8s pods。

With this architecture, we're able to support a huge set of streams and viewers, without origin cluster which is stateful and complex. Please note that edge only works for live streaming, such as RTMP/FLV. Other edge also works well, for example, HLS edge cluster works with Proxy, from where NGINX fetch and cache HLS. After WebRTC supports cascading, it also works with proxy.

中文：使用这种架构，可以支持很多流，也支持很多观看。我们并没有使用源站集群，因为源站集群是有状态的，因此很复杂。请注意Edge是为直播设计的，支持RTMP和FLV。当然HLS边缘集群也可以和Proxy工作得很好，NGINX可以从Proxy拉HLS流。如果WebRTC支持级联后，那么级联服务器也可以从Proxy获取流。

Proxy Mirrors

To extend stream concurrency, proxy can mirror streams to multiple origin servers, to enable you to play stream from different origin server, for fault tolerance and scale out cluster capability.

中文：为了支持单流的并发数目，Proxy可以讲流镜像到多个源站，你可以在多个不同的源站访问这个流，增加单个流的并发能力。

Proxy --+---> Origin Server A
        +---> Origin Server B
        +---> Origin Server C

For example, a RTC origin server could serve about 300 to 700 players for each stream, you can use proxy to mirror the same stream to 3 origin servers, to enable 900 to 2100 players.

中文：例如，单个RTC源站可能支持300到700个播放器拉流，可以用Proxy将流镜像到3个源站后，就可以支持900到2100个播放器访问。

Limitation

The limitation of proxy is the number of viewers for a stream, which should never exceed each single server's capacity, becuase proxy always proxy the same stream to the same backend server. For example, SRS support 5k viewers for RTMP/FLV, about 500 viewers for WebRTC, please test the capacity by srs-bench.

中文：Proxy的限制是流的播放的数量，不能超过单个服务器支持的播放数量，这是因为Proxy总是将同一个流代理到同一个服务器。比如SRS支持5k客户端播放RTMP/FLV，或者500客户端播放WebRTC，详细的播放数量的支持请使用srs-bench测试。

It's the responsibility of Cluster to support a large set of viewers, such as Edge Cluster for RTMP/FLV, or HLS Cluster for HLS. WebRTC doesn't support cluster now, please read wiki of WebRTC.

中文：支持很多播放客户端，是集群需要支持的功能。比如Edge Cluster支持很多RTMP/FLV客户端，HLS Cluster支持很多HLS客户端。WebRTC目前还没有集群能力，详细请看WebRTC相关的文档。

For most use scenario, Proxy is much simple and useful ability for load balancing, because there're a set of streams to serve but not too much, and there're also a set of viewers for each stream but not too much. For example, building a system for 1k streams and each stream has 1k viewers, the total connections of system is no more than 1m.

中文：大多数场景中，Proxy比集群更有用，因为大多是有一些流但不会特别多，每个流有一些播放但不是特别多。比如，支持1k个流，每个流有1k个观看，整个系统的连接数不超过100万。

Proxy also works with cluster, for example, if you have 1k streams and each stream has 100k FLV viewers, the architecture is like this:

中文：Proxy可以和Cluster一起工作，比如你有1k个流，每个流有10万观看，系统结构如下。

Publisher ---RTMP--> Proxy --> Origin Servers 
Origin Server --RTMP--> Proxy --RTMP--> Edge Cluster
Edge Cluster --RTMP--> Players

Keep in mind that proxy is design for Origin server, so there should always be proxy for a origin server, even for edge server to pull stream from. Proxy is another similar solution like origin cluster, but it's much simple and works for all protocols like RTMP/FLV/HLS/WebRTC/SRT, while origin cluster only works for RTMP.

中文：请记住Proxy是为Origin设计的，所以每个Origin都需要通过Proxy提供服务，哪怕是Edge集群回源时也需要通过Proxy。其实Proxy是和Origin Cluster相似的方案，不过Proxy要更简单而且支持所有的协议包括RTMP/FLV/HLS/WebRTC/SRT，而Origin Cluster目前只支持RTMP协议。

Notes

Proxy should be written by Go/C++, stable and simple. Should be C++, because we might use eBPF.

中文：Proxy可以使用Go/C++实现，这样无状态化更容易实现，稳定性也更高。可能用C++，因为要使用eBPF。

It's possible to directly forward IP packets by kernel, to make proxy less CPU.

中文：由于内核的不断完善，也可考虑非流量代理方式，直接让内核将流量转移到不同的进程。

Proxy is much simpler than cluster, because both proxy and origin server is stateless, which can be deploy by K8s Deployment.

中文：Proxy是比Cluster要简单非常多的，因为Proxy和Origin都是无状态服务，都可以用K8s的Deployment部署。

[qiantaossx]

Make some changes on top of frp?

TRANS_BY_GPT3

[not2007]

May I ask what proxy solutions are currently available for research and modification?

TRANS_BY_GPT3

[winlinvip]

Update progress: Proxy is essentially a load balancing problem, and eBPF is a suitable implementation.

In addition, it is more suitable to use libbpf in C++ rather than cilium/ebpf in Go, as there are significant differences between the two.

TRANS_BY_GPT3

[jinleileiking]

webrtc要给sdp，而lb后面的rs是不暴露的，好像没法解啊, 在考虑nodeport是否能解这个问题'

Translation: 'webrtc needs to provide sdp, but the rs behind lb is not exposed, it seems difficult to solve, considering if nodeport can solve this problem

TRANS_BY_GPT3

[jinleileiking]

ebpf is too complicated.

We currently implement HLS.

pub -> lb1 -> srs -> pub hook -> go server -> save pub pod ip to mysql

play -> lb2 -> go server -> get pod ip> get m3u8, ts from srs pod > return

Disadvantage, dependency on the center. Depending on the business, if a go server is added, it will become a somewhat complex system.

Advantage, simple, sufficient for short-term low traffic, add a local cache of pod IP.

TRANS_BY_GPT3

[qiantaossx]

For the business scenario of converting RTMP streams to WebRTC streams, since the source server itself has state, it cannot be deployed on K8S. Currently, our business can only be directly deployed on physical machines.

Can we consider solving the state issue of the source server by adding this proxy layer?
player -> load balancer -> proxy (forwarding HTTP and UDP packets) -> SRS source server.

Since this proxy layer is stateless, it can be easily deployed through deployment, exposing a unified load balancer and domain name externally.

The UDP load balancer does not modify the source IP and source port of the UDP packet (https://exampleloadbalancer.com/nlbudp_detail.html). The proxy can use this tuple to determine which backend SRS the packet should be forwarded to. @winlinvip, can it be used in this way?

TRANS_BY_GPT3

[winlinvip]

@qiantaossx Proxy is designed to recognize the content of the request and determine which stream the client is requesting, and then forward it to the backend server. Therefore, it can be used for the origin server of all protocols.

Additionally, regarding the issue of the origin server having state, it is actually not possible to achieve complete statelessness, but it can be avoided through retrying. For example, if the origin server goes down, retrying the push stream will select a new origin server, and retrying the pull stream can also locate this new origin server again.

This way, both the origin server and the proxy can be deployed without state, achieving a very simple operation and maintenance.

PS: The origin server cluster is actually stateless as well. It is just that the current configuration requires a single address, so typically the origin server cluster is considered stateful. However, if we improve it by obtaining other origin server addresses from an API, then it can also be considered stateless.

PS: The preliminary plan is to implement the proxy in SRS 6.0, and development is expected to start in early 2023. If you are interested, you are welcome to participate.

TRANS_BY_GPT3

[jinleileiking]

Recently, I have been watching livekit, which can be used as a reference for its implementation. The pod will find its own external address (through NAT) and then provide this address to SDP, thus completing the distribution of the WebRTC source station in the cluster.

TRANS_BY_GPT3

[winlinvip]

We have confirmed that we will implement a Proxy within the SRS Stack, and we have also experimented with SRT #3869 and architecture, which works well within the Proxy, just like other protocols.

The reason for not implementing it within SRS is that the Proxy will definitely utilize Redis to achieve statelessness, and the Proxy does not require strong streaming protocol processing capabilities. It is more about forwarding UDP or TCP packets, hence Go is the most suitable language for this purpose.

Originally, the SRS Stack was implemented using Go to enhance its capabilities with a Proxy, which can also cover scenarios that require the expansion of a single SRS Stack instance.

TRANS_BY_GPT4

[yuriploc]

Could be possible for the Proxy to have an API or protocol to allow other caching mechanisms beyond redis?

[winlinvip]

We are currently preparing to implement a Proxy within Oryx, which is developed in Go and has an integrated Redis service. Consequently, by default, we will only support Redis as the key-value store. Support for other key-value stores would require modifications to the Oryx codebase.

TRANS_BY_GPT4

[winlinvip]

Today, I looked at the cluster architecture of Redis, which is divided into several types:

• Standalone Redis, without clustering capabilities.
• Master-Slave: Expands read capabilities, similar to SRS's Forward. However, Redis supports the promotion of a Slave to Master, using Sentinel for failover, which SRS does not support.
• Proxy: Twemproxy implements hashing based on the URL and forwards the load to Redis nodes. This architecture is similar to the Proxy we are discussing here.
• Cluster: Redis has supported clustering since version 3.0, using the Gossip protocol, which is a MESH network structure connecting nodes, supporting up to 1000 nodes. This architecture has no central node, and the client usage is relatively simple. However, it cannot be used in streaming media because it requires modifications to the client.

Single Node Redis

Standalone version architecture, refer to link:

Start Redis:

docker run -it --rm --name redis -p 6379:6379 redis:5.0 redis-server

Test and verify:

docker exec -it redis redis-cli set k v
#OK

docker exec -it redis redis-cli get k
#"v"

The architecture used in Oryx is a standalone Redis setup, because Oryx itself is designed as a single-node architecture.

Master Slave Redis

To expand reading capabilities, a configuration consisting of one Master and multiple Slaves can be used, as referenced in link.

First, initiate the Master:

docker run -it --rm --name master -p 6379:6379 redis:5.0 redis-server

Obtain the IP of the Master, then start the Slave service:

IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master)
docker run -it --rm --name slave -p 6380:6379 redis:5.0 \
    redis-server --replicaof $IP 6379

Test and verify that data is written to the Master and read from the Slave:

docker exec -it master redis-cli set k v

docker exec -it slave redis-cli get k
#"v"

This architecture merely enhances the read capacity and is suitable for scenarios with minimal writing but a substantial amount of reading.

Sentinel for Master Slave Redis

When the Master goes down, Sentinel can be used to promote a Slave to Master, achieving automatic failover. For reference, see link.

First, start the Master and Slave services:

docker run -it --rm --name master -p 6379:6379 redis:5.0 redis-server

IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master)
docker run -it --rm --name slave -p 6380:6379 redis:5.0 \
    redis-server --replicaof $IP 6379

Next, create a Sentinel:

IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master)
cat <<EOF > /tmp/sentinel-26379.conf
port 26379
sentinel monitor mymaster $IP 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1
EOF

docker run -it --rm --name sentinel0 -p 26379:26379 \
    -v /tmp/sentinel-26379.conf:/etc/redis/sentinel.conf redis:5.0 \
    redis-sentinel /etc/redis/sentinel.conf

• "mymaster" is the name of the master node being monitored.
• "$IP" is the IP address of the master node, for example, 172.17.0.2.
• "6379" is the port of the master node.
• "2" is the minimum number of Sentinels that need to agree for a failover to occur.

Next, create another Sentinel:

IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master)
cat <<EOF > /tmp/sentinel-26380.conf
port 26380
sentinel monitor mymaster $IP 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1
EOF

docker run -it --rm --name sentinel1 -p 26380:26380 \
    -v /tmp/sentinel-26380.conf:/etc/redis/sentinel.conf redis:5.0 \
    redis-sentinel /etc/redis/sentinel.conf

Retrieve master information:

docker exec -it sentinel0 redis-cli -p 26379 sentinel masters
#    3) "ip"
#    4) "172.17.0.2"

Turn off the Master:

docker rm -f master

View the Sentinel logs:

#+switch-master mymaster 172.17.0.2 6379 172.17.0.3 6379

docker exec -it sentinel0 redis-cli -p 26379 sentinel masters
#    3) "ip"
#    4) "172.17.0.3"

You are able to access the redis, because slave has been switched to master:

docker exec -it slave redis-cli set k v

Restart the master container, and notice that it has become a Slave.

#* +convert-to-slave slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.3 6379

Note: Although a Slave can be promoted to a Master, starting a Slave directly without the Master being up will lead to failure.

Twemproxy Proxy for Redis

Server proxies are sharded, with multiple Proxies connected behind the proxy. A simple architecture example is Twemproxy, for reference see the link.

To simplify deployment, Twemproxy is directly connected to the Redis master. First, create two masters:

docker run -it --rm --name master0 -p 6379:6379 redis:5.0 redis-server
docker run -it --rm --name master1 -p 6380:6379 redis:5.0 redis-server

Create a Twemproxy image:

cat <<EOF > /tmp/Dockerfile.twemproxy
FROM ubuntu:focal

RUN apt-get update -y && \
  apt-get install -y git build-essential autoconf automake libtool
  
WORKDIR /g
RUN git clone https://github.com/twitter/twemproxy.git && \
  cd twemproxy && autoreconf -fvi && ./configure && make

WORKDIR /g/twemproxy
CMD ["./src/nutcracker", "-c", "/etc/nutcracker.yml"]
EOF

docker build -t twemproxy:latest . -f /tmp/Dockerfile.twemproxy

Generate a configuration file and start the service:

IP0=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master0)
IP1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' master1)
cat <<EOF > /tmp/twemproxy.yml
alpha:
  listen: 0.0.0.0:22121
  hash: fnv1a_64
  distribution: ketama
  auto_eject_hosts: true
  redis: true
  server_retry_timeout: 2000
  server_failure_limit: 1
  servers:
   - $IP0:6379:1
   - $IP1:6379:1
EOF

docker run --rm -it --name proxy -p 22121:22121 \
    -v /tmp/twemproxy.yml:/etc/nutcracker.yml twemproxy:latest

Verify the outcome:

redis-cli -p 22121 set k v
#OK
redis-cli -p 22121 get k
#"v"

redis-cli -p 6380 get k
#"v"

It can be observed that stopping this service on master1 will result in a failure to retrieve.

redis-cli -p 22121 get k
#(error) ERR No route to host

Thus, the logic of this Proxy is to lock onto the target service based on the key k.

Cluster for Redis

Redis supports clustering with a MESH topology where instances are interconnected. For reference, consult the official documentation and this article. For the cluster protocol, refer to this link.

For an understanding of the principles behind Redis clustering, refer to the Gossip protocol.

Start at least six nodes, with three acting as Master nodes and each having one Slave node for backup purposes:

docker run -it --rm --name cluster0 -p 6379:6379 redis:5.0 redis-server \
    --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes
docker run -it --rm --name cluster1 -p 6380:6380 redis:5.0 redis-server \
    --port 6380 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes
docker run -it --rm --name cluster2 -p 6381:6381 redis:5.0 redis-server \
    --port 6381 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes
docker run -it --rm --name cluster3 -p 6382:6382 redis:5.0 redis-server \
    --port 6382 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes
docker run -it --rm --name cluster4 -p 6383:6383 redis:5.0 redis-server \
    --port 6383 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes
docker run -it --rm --name cluster5 -p 6384:6384 redis:5.0 redis-server \
    --port 6384 --cluster-enabled yes --cluster-config-file nodes.conf \
    --appendonly yes

Check if the service has started properly:

redis-cli info server
#redis_mode:cluster

redis-cli cluster nodes
#c47b6a17b1aac4dc0fec5953af06aafcf8e46c3d :6379@16379 myself,master - 0 0 0 connected

Create a cluster with the first three nodes as Master nodes and the subsequent three nodes as Slave nodes:

IP0=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster0)
IP1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster1)
IP2=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster2)
IP3=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster3)
IP4=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster4)
IP5=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster5)

docker exec -it cluster0 redis-cli \
    --cluster create $IP0:6379 $IP1:6380 $IP2:6381 $IP3:6382 $IP4:6383 $IP5:6384 \
    --cluster-replicas 1
#Master[0] -> Slots 0 - 5460
#Master[1] -> Slots 5461 - 10922
#Master[2] -> Slots 10923 - 16383
#Adding replica 172.17.0.6:6383 to 172.17.0.2:6379
#Adding replica 172.17.0.7:6384 to 172.17.0.3:6380
#Adding replica 172.17.0.5:6382 to 172.17.0.4:6381

View the cluster:

redis-cli  cluster nodes

#703552c3d6655e6a85fb90703a96ac9ac5c95605 172.17.0.2:6379@16379 myself,master - 0 1716539138000 1 connected 0-5460
#7640460fd16a07cf96c963b7be3e92f7dec75c43 172.17.0.3:6380@16380 master - 0 1716539139000 2 connected 5461-10922
#52c6d4d44c6bbdda13960f6771dd294ec8c8197f 172.17.0.4:6381@16381 master - 0 1716539137000 3 connected 10923-16383

#a9c95ef4575de5a75c4b60b41a397bca7c750156 172.17.0.6:6383@16383 slave 703552c3d6655e6a85fb90703a96ac9ac5c95605 0 1716539137000 5 connected
#87e76c7efb0aef76d64db45d4bc7ed73d8838a19 172.17.0.7:6384@16384 slave 7640460fd16a07cf96c963b7be3e92f7dec75c43 0 1716539139856 6 connected
#91d697f493a7757d610c18fa9bcbffedb4a3570a 172.17.0.5:6382@16382 slave 52c6d4d44c6bbdda13960f6771dd294ec8c8197f 0 1716539137831 4 connected

Set up a key-value pair; this requires executing within the container because it returns the container's IP, which is similar to a 302 response:

docker exec -it cluster0 redis-cli set k v
#(error) MOVED 7629 172.17.0.3:6380

docker exec -it cluster0 redis-cli -c set k v
#OK

Note: When executing with the redis-cli -c command, it will automatically resolve the MOVED error by redirecting to the correct host and port and then re-executing the command.

Consistent Hashing

To balance keys across a group of servers, you can use a hash algorithm to assign them to servers. To avoid key migration caused by adding or removing servers, you can expand the base to uint32, making the server positions fixed. To avoid load imbalance, you can add virtual nodes, which means replicating the same node multiple times. Refer to the provided link for more details.

Hash Slot

Hash Slot, or Hash Slots, involves assigning a range of hash values to servers. For example, in a cluster with 3 nodes, Node A contains hash slots from 0 to 5500, Node B contains hash slots from 5501 to 11000, and Node C contains hash slots from 11001 to 16384. Refer to the provided link for more details.

Instead of using Consistent Hashing, Hash Slot provides a more straightforward way to distribute the load. Consistent Hashing is more complex to implement. When a node is removed, data is reassigned to the next node in Consistent Hashing, but with Hash Slot, data can be flexibly reassigned to multiple other nodes.

TRANS_BY_GPT4

[winlinvip]

Proxy server: #4158

SRS heartbeat register for proxy server: #4171

Document: https://ossrs.io/lts/en-us/docs/v7/doc/origin-cluster