Allow Batch Publishing For Applications

[nisdas]

For a more detailed reasoning on the motivation for this:
https://ethresear.ch/t/improving-das-performance-with-gossipsub-batch-publishing/21713

This pull request adds in a new publishing option that allows messages to be published as a batch instead of being queued and sent out individually. Why would this be desired ? Applications can send out multiple messages at the same time. In the event a node has constrained upload bandwidth and these messages are large, the router would take up a longer than desired time sending out D copies to all mesh peers for the first message published. This can add a non-trivial amount of latency for the other messages being propagated to the rest of the network.

This option allows these messages and their copies to be shuffled randomly and then queued in a random order and sent out to their respective peers. This allows for the first copy of any message in the batch to be propagated to the rest of the network much faster. With a first copy being sent out quicker, this leads to earlier message arrival time for the whole network.

• Add Batch Publishing Option.
• Add Batch Message Object to track all the desired message ids and the rpc messages to be published to individual peers.
• Allow Publishing of the whole batch once it is complete.
• Add Tests that the feature works as expected.

[vyzo]

I think the latency would actually increase, as the whole rpc message has tobe received and decoded before any message is processed, no?

[nisdas]

I think the latency would actually increase, as the whole rpc message has tobe received and decoded before any message is processed, no?

Maybe I am missing something, why would that be the case ? This PR just allows you to withhold sending the rpc message to your mesh/direct peers till all the batch message ids have been processed by the gossip router. We then just shuffle the rpcs and send them out so that each message in the batch is sent out 'fairly'

[vyzo]

ah you are not sending as one rpc, so it should be ok.

[MarcoPolo]

Maybe we want to call RemoveMessage earlier. I think there's an issue where if your gs.p.topics[topic] is empty then RemoveMessage will never be called and the whole batch will never send. You can hit this if you are batching messages across topics and one topic happens to have no peers.

[MarcoPolo]

I find this API a little awkward to use. You have to queue up the messages with the batched message directly, then pass in batched message on a publish call to every message you queued up. It also forces you to have access to the msg id function at batching time, which can be annoying.

What about flipping the api a bit? Using only BatchedMessage as your entrypoint for batching and publishing. It can take care of calculating the msg id for the given message and making sure to publish all queued messages. Something like this:

batchedMsg := pubsub.NewBatchMessage()

for _, msg := range msgsToPublish {
	batchedMsg.Add(topic, msg)
}

batchedMsg.Publish()