The VMware research team's SBFT algorithm achieved 50 I/Os per second when processing four months worth of Ethereum transactions (a 10x speedup).
This statement may not appear noteworthy in the context of how many I/Os per second VISA processes. It is, however, a significant advancement in the art of consensus algorithms, and a promising step in the right direction for enterprise-class blockchains.
I've spent the last few weeks blogging about the fact that new trusted storage features perform slowly at first and then improve over time. My thoughts included a look at the latest evolution of a trusted storage platform: blockchain.
One of the problems with blockchain performance is the speed of the underlying consensus algorithm. In a recent post, I highlighted common scaling problems for consensus algorithms. I ended the article by mentioning VMware's blockchain research team has recently made some breakthroughs with their Scalable Byzantine Fault Tolerant (SBFT) consensus algorithm.
Their latest publication is an example of the inevitable progress that the research community will make as they harden blockchain trust and performance. The publication is not just a white paper; it represents working code. It will only be a matter of time before VMware's work appears in product form.
I've studied the consensus advancements described in their paper. What are the improvements that SBFT brings to the table? For me, several highlights stand out.
Running in a Permissioned Context
One of VMware's strengths is their ability to create deterministic, enterprise-class network connections via technologies such as NSX. Many enterprise customers would like to deploy blockchain applications in this type of permissioned environment (as opposed to a public environment like Bitcoin or public Ethereum). The beauty of VMware network configurations is that they allow network administrators to establish an upper bound on transmission delays. If a request between nodes exceeds this upper-bound, then the network can assume that the node has either disappeared or has experienced a failure.
VMware's researchers have been problem-solving in this type of deterministic environment.
PBFT as Algorithmic Foundation
The consensus improvements realized by VMware's researchers have their foundation in a Partial Byzantine Fault Tolerance (PBFT) approach. PBFT is a leader-based algorithm with two phases. An all-to-all message exchange characterizes each stage. In a previous post I inserted a diagram from a USENIX paper and then highlighted all-to-all messaging exchanges.
These messaging exchanges, showed here for four nodes, become highly problematic as blockchain configurations scale to hundreds of nodes. As the amount of messages scale quadratically, the system runs the risk of losing liveness (continual progress while processing new requests).
Removing the problem of quadratic scale is not straightforward, and VMware researcher Dahlia Malkhi has shared her thoughts on how solving it is trickier than one might think.
The SBFT solution has three basic components.
1. Using the Right Amount of Batching
The messaging approach highlighted by PBFT above occurs for every consensus decision. Grouping minimizes messaging, but batching too many transactions together can introduce unacceptable delays (e.g., the first transaction in a group waits the longest). SBFT approaches batching in the following way:
"SBFT strikes a balance between throughput and latency with mild batching only, a few dozens of transactions, maintaining latency under 60 milliseconds."
2. Improving the Zyzzyva Approach
In addition to implementing batching, SBFT adopts the Zyzzyva approach to minimize consensus messaging in the standard case (i.e., all nodes are healthy). The method depicted below (from the USENIX article) makes it clear that all-to-all (quadratic) messaging does not occur with Zyzzyva.
The paper highlights the benefit of using the Zyzzyva approach:
"Embracing this idea, SBFT is a PBFT variant that incorporates into the protocol an optimistically fast track with linear cost during synchronous, faultless periods.
SBFT leverages the idea of linear scale in the optimum (healthy) case, but it also addresses (and fixes) some of the flaws in Zyzzyva.
3. Single Confirmation to Clients
The third improvement provided by SBFT is a significant reduction in the number of confirmations back to the client (i.e., the application that wishes to write a new transaction into a blockchain). The diagram below highlights that a client receives multiple messages from multiple replicas for Zyzzyva, but only one with SBFT.
The minimization of client messages is made possible by using a technique known as threshold signatures. This approach combines messages from rotating collectors (the C-collector and E-collector displayed above) and combines all signatures into one. While this introduces additional computational complexity at the collector nodes, it also has the advantage of (a) a single client message (CPU-friendly for application developers), and (b) a single verification needed by replicas (network friendly for blockchain nodes).
The paper describes how it uses two different threshold signature techniques (BLS for < 70 nodes, and RSA >= 70). The paper states that the use of threshold signatures in this way is unique in the industry.
"no deployed BFT system we are aware of sends only a single confirmation to clients. Not only that, but repeated queries of the same value can return a single response as well."
What's Next for SBFT?
The results published by the VMware research team (and validated by working SBFT code) are being shared both externally with the industry and internally within VMware and Dell Technologies.
Members of Dell Technologies' Blockchain Steering team will be presenting results from the paper and other Dell Blockchain points of view at several upcoming Dell Technologies World sessions.
Wednesday, May 2nd 8:30 AM: Thinking in Blockchain: How to Apply Blockchain Technology to Business Processes.
Wednesday, May 2nd 12:00 PM: Blockchain Challenges and Dell Solutions.
Wednesday, May 2nd 1:30 PM: Advancements in Blockchain Development.
Many thanks to the VMware blockchain research team for educating me on SBFT!
Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd