I've written a few posts describing the basics of a data analytic stack, including the CloudFoundry layer, and the mobile layer.
The main value proposition of Pivotal's stack, as I've written about previously, is software development speed for data analytic applications, as represented by the lifecycle diagram below.
For this post I'd like to dive into the top-most layer of Pivotal's stack, which is often depicted as follows:
At first glance it seems obvious that frameworks like node.js and Rails can help speed software development time. They are spoken of as agile, fast-paced, iterative, etc. Spring can be viewed the same way. I have looked at this layer and wondered if there was any more to the message other than "Pivotal's Data Fabric can be consumed by agile frameworks, which therefore improves software development speed".
As I often do in the case where I'd like a deeper dive into frameworks like Spring, I turned to Pivotal's Brian Dussault (I recommend you follow him on Twitter).
Brian cleared up a lot of my questions about the Application Fabric by pointing out a few things:
- The Spring IO Platform is the premier technology to build next generation applications on the JVM
- Pivotal’s App Fabric is modern cloud-scale middleware (e.g RabbitMQ, Reddis, tc Server) that powers some of the biggest cloud services in the world (e.g. TravisCI) with some of the most demanding workloads (e.g. one million compex events per second).
- App Fabric, combined with the Spring IO Platform, enables organizations to collect, process and deliver information to customers
- The PivotalOne/CF build-packs support numerous development frameworks ~ Spring, Rails, etc
- Spring is owned by Pivotal, so enterprise customers using Spring will have a seamless experience deploying Spring workloads to PivotalOne/CF.
Brian believes that this can be best understood in the context of a fairly standard Big Data use case for developers:
- Ingest data at high speed
- Store the data in high speed memory (e.g. the GemFire layer) for realtime insights and micro-batching to HDFS
- Persist the data into HDFS
- Run the analytics (e.g. Hadoop, SQL)
- Export the results for online apps (mobile or web) to display to end customers
So how can the Spring IO Platform accelerate building applications for this use case? According to Brian, here's how:
- Gemfire doesn't have an application framework to open up a tcp port and receive a tcp stream. This is a good case for Spring XD which has adapters (out of the box functionality) to recieve a stream of data and persist to Hadoop (steps 1-3 from above use case) above. Without Spring this would all be custom, home grown code. This is what enables developer agility.
- Running analytics on Hadoop often means that a developer needs to create and orchestrate some mapreduce jobs and store the results in the HDFS. The Spring IO Platform once again has out of the box functionality to handle this use case (#4). This of course can be combined with Data Scientist experise and analysis
- (Step 5 from above) Once all the big data analysis is finished the "reduced" data typically makes its way out of HDFS to traditional databases (Oracle, SQL Server, etc) to be used by online applications. The Spring IO Platform also provides capabilities to simplify this effort. Of course there are many back-office use cases for big data that don't require delivering insight back to the user, but this is just one common pattern
- (Beyond…)The Spring IO Platform goes beyond big data and provides rich features for mobile, REST, and web application development; allowing organizations to deliver data insight back to customers on any device where and when they need it.
The Spring IO Platform represents a major advancement in developer productivity and complements Pivotal’s multi-latency data stack. PivotalOne combines data and applications into a single platform, enabling agile development teams to rapidly update and scale applications across public and private clouds. PivotalOne ensures that organizations can bring new solutions to market faster than ever before. It’s the end-to-end agility (development through deployment) that differentiates Pivotal’s portfolio from the rest of the market.
To learn more about what kind of Spring functionality is available, I recommend the SpringCentral twitter feed and the Spring blog.
Steve
Twitter: @SteveTodd
EMC Fellow
Comments