I recently finished writing a series of articles discussing next-generation applications and the new models of IT infrastructure that are often required to support them. My focus was primarily on the building blocks used in the new infrastructure.
I’d like to turn my attention now to the application side of the discussion. I’ve mentioned several times that one of the key value propositions of Pivotal is the reduction of software development time, enabling companies to deliver better software experiences to their customers, at a pace faster than their competitors.
Pivotal is trying to reduce application development time from months down to days (and even hours). This model is akin to what VMware did with driving down IT provisioning times (it’s no coincidence that Pivotal as a company was created via a significant collection of VMware software assets).
Let’s start with the following use case:
- A corporation has an idea for a business app that could drive significant revenue if it crosses over the 300,000 concurrent user limit within 2-3 months.
- As the app scales it needs to allow users to use social messaging via an eventually consistent model.
- Any downtime and/or quality issues would be the kiss of death in user adoption.
- Scaling to 1,000,000 concurrent users is the plan.
In other words, the corporation wants the app to be wildly successful in a short amount of time and therefore it needs a scalable, high quality messaging framework that knows how to start small and go big fast.
If you look around the industry, there’s really only one choice: Pivotal RabbitMQ. RabbitMQ is used by many of the 'consumer grade' companies that operate in the manner to which many enterprises aspire. Consider the list of users below as a proof point.
If you’re Googling, Facebooking, Tivoing, reading the news, booking travel or doing pretty much anything else online, chances are you’re using a system that involves RabbitMQ. In fact, it’s hard not to spend a day online without using a service that depends on it.
RabbitMQ is also part of one of the biggest, active IT projects in the world: UIDAI – the Unique Identification Authority of India (first described by VMware here). This project, which is being implemented by Aadhar, currently has 1.3 Billion end users! So if you want to dress your application for success at scale, RabbitMQ is a proven product.
The goal of the UIDAI application is to enable social services and microfinance for over 1+ billion residents. It’s the biggest online identity project in the world, and RabbitMQ is part of a very significant, country-wide IT infrastructure with a lot of moving parts. Whenever moving parts require coordination, RabbitMQ is the reliable messaging framework to deliver it.
The Indian government has a strategy overview and several other interesting documents describing the initiative here and here.
If a new enterprise application does experience wild success, the choice of RabbitMQ as the messaging framework will free the application to scale dramatically. As I’ve started to study the reasons for RabbitMQ’s success, I’ve learned a number of key insights from Helena Edelson’s blog (with a portion cut-and-pasted below):
- It’s wicked fast and reliable
- Supports security and transactions, persistence, client acks, and flow control
- Widely proven and used heavily on Amazon EC2
- It is open source with support services and several very active user lists
- Hugely interoperable
- RabbitMQ brokers run on many platforms and offer several clients (Java, .net and others), all which speak AMQP protocol to each other. Anyone can write a RabbitMQ client in any language: Ruby, Java, Spring, Python, C, C#, Perl, Erlang, Lisp, Haskell, PHP, _your_language_here.
- Easily clusterable
- Works with several protocols: AMQP, STOMP, XMPP, 0MQ
- Highly-flexible topologies
- Can work with Pacemaker and associated tools to provide High Availability
- Supports several extension points via plug-ins, e.g. exchange types and backing queue storage
- Provides management and monitoring of brokers
- vFabric Hyperic management of RabbitMQ
- RabbitMQ command line tool and plugin for real-time activity and management
- Messaging services hosted in the cloud with RabbitMQ as part of overall cloud infrastructure services are highly-supported
Going forward you will witness RabbitMQ messaging and the Spring framework driving the Industrial Internet and Internet of Things (IoT), serving as bridge from sensors to both existing and new applications:
What makes this possible is that RabbitMQ supports a range of messaging protocols, including MQTT, the emerging standard to send messages to and from sensors and mobile devices; JMS, used by many legacy Java apps built in the 90s and 00s; and AMQP, developed several years ago and increasingly used for server-to-server messaging.
I wouldn’t want to leave the impression that RabbitMQ and Spring only run in a “3rd platform” context. The truth is that their ability to run on 1st, 2nd, 3rd, and Industrial Internet platforms is a key reason for choosing them. They are the bridge that facilitates interoperability and/or migration between different infrastructures. It’s imperative to build application interoperability at the topmost layer as well as application data mobility at the bottom-most layers. As the diagram below shows, RabbitMQ is accessible by languages spanning four decades of the IT industry, from some of the oldest (Cobol) to some of the newest (Go). As well some of the most popular, like Java, C#, C, Ruby, and Python.
As an example, you can have a Cobol application streaming messages into an Android or iPhone app using websockets. Very few products allow developers to interoperate between IT platforms from different eras.
In upcoming posts I will take a deeper look at how Pivotal’s RabbitMQ and Spring are driving new applications in for the Industrial Internet. In the meantime, here are some additional examples of scale:
- RabbitMQ Scales to 74,000 Continuous Integration Builds a Day with TravisCI.
- See how C24 was able to move from 20K to 1 MM complex event processed per second
- Examples of how companies like SoundCloud and SocialVane architected their messaging
Steve
Twitter: @SteveTodd
EMC Fellow
Comments