Looks cool, thank you burntsushi for this. I have similar complaints about existing date-time libraries in rust. I’ll replace chrono/time with Jiff in my projects.

They can replace Void with AppendEntriesResponse anytime they want and that won't break anything since the change is wire compatible.

Didn't know that! (Good to know, and thanks!)

This is awesome! Can’t wait to try it out.

One question: I assume RedPanda using raft to replicate the topic content not just metadata. Is that correct? If so, how does it perform better compared to Kafka’s ISR? Since raft might be slow for this kind of workload. If I remember correctly, Liftbridge was using raft for log replication and switched away from that because of the performance problems.

Checkout this _extremely_ detailed work that explains exactly that - https://vectorized.io/kafka-redpanda-availability/

To quote my colleague Denis - The results fit the theory. Raft’s (and Redpanda’s) performance is proportional to the best of the majority of nodes, while sync replication (Kafka) works only as well as its worst-performing node.

AFAIK we can push the limits of hardware on throughput.

The gist is that I'd need more details exactly on what you mean raft is slow.

We support the same level of acks as kafka except acks=-1 is much stronger gurantees due to the log completeness guarantee of raft.

for 0 and 1 we short circuit the raft acknowledgement and return to the client to match the exepectations of acknowledgements come to be known by users.

There should be no perf penalty vis-a-vis kafka in any setting I can think of. If it is, is probably a bug on our side.

Outstanding article!

Are you hoping to achieve no-downtime leader kills by going with the latest Paxos improvements, as you mention at the end?

So many goodies to come.

First we need to finish multi partition transactions that we don't yet support.

We need to balance eager cluster balancing based on saturation (network, cpu, disk, etc) with other performance improvements.

As is the cluster achieves steady state in a couple of seconds for affected partitions and clients in Kafka retry.

It's interesting to see that there are still opportunities for further improvements in raft. Thanks for the reply!

Then, you might be interested in watching this: https://www.youtube.com/watch?v=Pz-4co8IaI8

Nice article. Could someone suggest me a book that explains this stuff further?