Click here to close now.

Welcome!

AJAX & REA Authors: Pat Romanski, Liz McMillan, VictorOps Blog, Elizabeth White, Ken Fogel

Related Topics: Java, XML, SOA & WOA, Eclipse, AJAX & REA, Apache

Java: Article

The Disruptor Framework: A Concurrency Framework for Java

Rediscovering the Producer-Consumer Model with the Disruptor

Let's start with the basic question: What is the disruptor? The disruptor is a concurrency framework for Java that allows data sharing between threads. The age old way of coding a producer-consumer model is to use a queue as the buffer area between the producer and the consumer, where the producer adds data objects to the queue, which are in turn processed by the consumer. However, such a model does not work well at the hardware level and ends up being highly inefficient. The disruptor in its simplest form replaces the queue with a data structure known as the ‘ring buffer'. Which brings us to the next question, what is the ring buffer? The ring buffer is an array of fixed length (which must be a power of 2), it's circular and wraps. This data structure is at the core of what makes the disruptor super fast.

Let's explore a simple everyday scenario in enterprise architectures. A producer (let's call it the publisher) creates data and stores it in the queue. Two immediate consumers (let's call them fooHandler and barHandler) consume the data and make updates to it. Once these 2 processors are done with a piece of data, it is then passed on to a third consumer (let's call it fooBarHandler) for further processing. In a concurrent processing system using legacy techniques, coding this architecture would involve a crisscross of queues and numerous concurrency challenges, such as dealing with locks, CAS, write contention, etc. The disruptor on the other hand immensely simplifies such a scenario by providing a simple API for creating the producer, consumers and ring buffer, which in turn relieve the developer of all concerns surrounding handling concurrency and doing so in an efficient manner. We shall now explore how the disruptor works its magic and provides a reliable messaging framework.

Writing to the ring buffer

Looking at the figure above, we find ourselves in the middle of the action. The ring buffer is an array of length 4 and is populated with data items - 4,5,6 and 7, which in the case of the disruptor are known as events. The square above the ring buffer containing the number 7 is the current sequence number, which denotes the highest populated event in the ring buffer. The ring buffer keeps track of this sequence number and increments it as and when new events are published to it. The fooHandler, barHandler and fooBarHandler are the consumers, which in disruptor terminology are called ‘event processors'. Each of these also has a square containing a sequence number, which in the case of the event processors denotes the highest event that they have consumed/processed so far. Thus its apparent that each entity (except the publisher) tracks its own sequence number and thus does not need to rely on a third party to figure out which is the next event its after.

The publisher asks the ring buffer for the next sequence number. The ring buffer is currently at 7, so the next sequence number would be 8. However, this would also entail overwriting the event with sequence number 4 (since there are only 4 slots in the array and the oldest event gets replaced with the newest one). The ring buffer first checks the most downstream consumer (fooBarHandler) to determine whether it is done processing the event with sequence number 4. In this case, it has, so it returns the number 8 to the publisher. In case fooBarHandler was stuck at a sequence number lower than 4, the ring buffer would have waited for it to finish processing the 4th event before returning the next sequence number to the publisher. This sequence number helps the publisher identify the next available slot in the ring buffer by performing a simple mod operation. indexOfNextAvailableSlot = highestSeqNo%longthOfRingBuffer, which in this case is 0 (8%4). The publisher then claims the next slot in the ring buffer (via a customizable strategy depending on whether there is a single or multiple publishers), which is currently occupied by event 4, and publishes event 8 to it.

Reading from the ring buffer by immediate consumers

The figure above shows the state of operations after the publisher has published event 8 to the ring buffer. The ring buffer's sequence number has been updated to 8 and now contains events 5,6,7 and 8. We see that foohandler, which has processed events upto 7, has been waiting (using a customizable strategy) for the 8th event to be published. Unlike the publisher though, it does not directly communicate with the ring buffer, but uses an entity known as the ‘sequence barrier' to do so on its behalf. The sequence barrier let's fooHandler know that the highest sequence number available in the ring buffer is now 8. FooHandler may now get this event and process it.

Similarly, barHandler checks the sequence barrier to determine whether there are any more events it can process. However, rather than just telling barHandler that the next (6th) event is up for grabs, the sequence barrier returns the highest sequence number present in the ring buffer to barHandler too. This way, barHandler can grab events 6,7,8 and process them in a batch before it has to enquire about further events being published. This saves time and reduces load.

Another important thing to note here is that in the case of multiple event processors, any given field in the event object must only be written to by any one event processor. Doing so prevents write contention, and thus removes the need for locks or CAS.

Reading from the ring buffer by downstream consumers

A few moments after the set of immediate consumers grab the next set of data, the state of affairs looks like the figure above. fooHandler is done processing all 8 available events (and has accordingly updated its sequence number to 8), whereas barHandler, being the slow coach that it is, has only processed events upto number 6 (and thus has updated sequence number to 6). We now see that fooBarHandler, which was done processing events upto number 5 at the start of our examination, is still waiting for an event higher than that to process. Why did its sequence barrier not inform it once event 8 was published to the ring buffer? Well, that is because downstream consumers don't automatically get notified of the highest sequence number present in the ring buffer. Their sequence barriers on the other hand determine the next sequence number they can process by calculating the minimum sequence number that the set of event processors directly before them have processed. This helps ensure that the downstream consumers only act on an event once its processing has been completed by the entire set of upstream consumers. The sequence barrier examines the sequence number on fooHandler (which is 8) and the sequence number on barHandler (which is 6) and decides that event 6 is the highest event that fooBarHandler can process. It returns this info to fooBarHandler, which then grabs event 6 and processes it. It must be noted that even in the case of the downstream consumers, they grab the events directly from the ring buffer and not from the consumers before them.

Well, that is about all you would need to know about the working of the disruptor framework to get started. But while this is all well and good in theory, the question still remains, how would one code the above architecture using the disruptor library? The answer to that question lies below.

Coding the disruptor

public final class FooBarEvent {
private double foo=0;
private double bar=0;
public double getFoo(){
return foo;
}
public double getBar() {
return bar;
}
public void setFoo(final double foo) {
this.foo = foo;
}
public void setBar(final double bar) {
this.bar = bar;
}
public final static EventFactory<FooBarEvent> EVENT_FACTORY
= new EventFactory<FooBarEvent>() {
public FooBarEvent newInstance() {
return new FooBarEvent();
}
};
}

The class FooBarEvent, as the name suggests, acts as the event object which is published by the publisher to the ring buffer and consumed by the eventProcessors - fooHandler, barHandler and fooBarHandler. It contains two fields ‘foo' and ‘bar' of type double, along with their corresponding setters/getters. It also contains an entity ‘EVENT_FACTORY' of type EventFactory, which is used to create an instance of this event.

public class FooBarDisruptor {           
public static final int RING_SIZE=4;
public static final ExecutorService EXECUTOR
=Executors.newCachedThreadPool();

final EventTranslator<FooBarEvent> eventTranslator
=new EventTranslator<FooBarEvent>() {
public void translateTo(FooBarEvent event,
long sequence) {
double foo=event.getFoo();
double bar=event.getBar();
system.out.println("foo="+foo
+", bar="+bar
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> fooHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double foo=Math.random();
event.setFoo(foo);
System.out.println("setting foo to "+foo
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> barHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double bar=Math.random();
event.setBar(bar);
System.out.println("setting bar to "+bar
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> fooBarHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double foo=event.getFoo();
double bar=event.getBar();
System.out.println("foo="+foo
+", bar="+bar
+" (sequence="+sequence+")");
}
};

public Disruptor setup() {
Disruptor<FooBarEvent> disruptor =
new Disruptor<FooBarEvent>(FooBarEvent.EVENT_FACTORY,
EXECUTOR,
new SingleThreadedClaimStrategy(RING_SIZE),
new SleepingWaitStrategy());
disruptor.handleEventsWith(fooHandler, barHandler).then(fooBarHandler);
RingBuffer<FooBarEvent> ringBuffer = disruptor.start();             
return disruptor;
}

public void publish(Disruptor<FooBarEvent> disruptor) {
for(int i=0;i<1000;i++) {
disruptor.publishEvent(eventTranslator);
}
}

public static void main(String[] args) {
FooBarDisruptor fooBarDisruptor=new FooBarDisruptor();
Disruptor disruptor=fooBarDisruptor.setup();
fooBarDisruptor.publish(disruptor);
}
}

The class FooBarDisruptor is where all the action happens. The ‘eventTranslator' is an entity which aids the publisher in publishing events to the ring buffer. It implements a method ‘translateTo' which gets invoked when the publisher is granted permission to publish the next event. fooHandler, barHandler and fooBarHandler are the event processors, and are objects of type ‘EventHandler'. Each of them implements a method ‘onEvent' which gets invoked once the event processor is granted access to a new event. The method ‘setup' is responsible for creating the disruptor, assigning the corresponding event handlers, and setting the dependency rules amongst them. The method ‘publish' is responsible for publishing a thousand events of the type ‘FooBarEvent' to the ring buffer.

In order to get the above code to work, you must download the disruptor jar file from http://code.google.com/p/disruptor/downloads/list and include the same in your classpath.

Conclusion
The disruptor is currently in use in the ultra efficient LMAX architecture, where it has proven to be a reliable model for inter thread communication and data sharing, reducing the end to end latency to a fraction of what queue based architectures provided. It does so using a variety of techniques, including replacing the array blocking queue with a ring buffer, getting rid of all locks, write contention and CAS operations (except in the scenario where one has multiple publishers), having each entity track its own progress by way of a sequence number, etc. Adopting this framework can greatly boost a developer's productivity in terms of coding a producer-consumer pattern, while at the same time aid in creating an end product far superior in terms of both design and performance to the legacy queue based architectures.

More Stories By Sanat Vij

Sanat Vij is a professional software engineer currently working at CenturyLink. He has vast experience in developing high availability applications, configuring application servers, JVM profiling and memory management. He specializes in performance tuning of applications, reducing response times, and increasing stability.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
SYS-CON Media announced today that XebiaLabs launched a popular blog feed on DevOps Journal with close to 2,000 story reads in less than a day. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. DevOps Journal brings valuable information to DevOps professionals who are transforming the way enterprise IT is done.
Cloudian, Inc., the leading provider of hybrid cloud storage solutions, today announced availability of Cloudian HyperStore 5.1 software. HyperStore 5.1 is an enhanced Amazon S3-compliant, plug-and-play hybrid cloud software solution that now features full Apache Hadoop integration. Enterprises can now transform big data into smart data by running Hadoop analytics on HyperStore software and appliances. This in-place analytics, with no need to offload data to other systems for Hadoop analyses, en...
Since 2008 and for the first time in history, more than half of humans live in urban areas, urging cities to become “smart.” Today, cities can leverage the wide availability of smartphones combined with new technologies such as Beacons or NFC to connect their urban furniture and environment to create citizen-first services that improve transportation, way-finding and information delivery. In her session at @ThingsExpo, Laetitia Gazel-Anthoine, CEO of Connecthings, will focus on successful use c...
XebiaLabs has announced record growth and major highlights for 2014. These milestones include: Triple-digit worldwide revenue growth. Record number of new customers including 3M, Allianz, American Express, Credit Agricole, Digital Globe, Electronic Arts, EMC, Expedia, Fandango / NBC Universal, GATX, General Electric, ING, KPMG, Liberty Mutual, Natixis, Paychex, Providence Health, Societe General, Thomson Reuters TIIA-CREF and Umpqua Bank.
CodeFutures has announced Dan Lynn as its new CEO. Lynn assumes the role from Founder Cory Isaacson, who has joined RMS and will now serve as chairman of CodeFutures. Lynn brings more than 14 years of advanced technology and business success experience, and will help CodeFutures build on its industry leadership around its Agile Big Data initiatives. His technical expertise will be invaluable in advancing CodeFutures’ AgilData platform and new processes for streamlining and gaining value from gro...
The speed of software changes in growing and large scale rapid-paced DevOps environments presents a challenge for continuous testing. Many organizations struggle to get this right. Practices that work for small scale continuous testing may not be sufficient as the requirements grow. In his session at DevOps Summit, Marc Hornbeek, Sr. Solutions Architect of DevOps continuous test solutions at Spirent Communications, will explain the best practices of continuous testing at high scale, which is r...
The explosion of connected devices / sensors is creating an ever-expanding set of new and valuable data. In parallel the emerging capability of Big Data technologies to store, access, analyze, and react to this data is producing changes in business models under the umbrella of the Internet of Things (IoT). In particular within the Insurance industry, IoT appears positioned to enable deep changes by altering relationships between insurers, distributors, and the insured. In his session at @Things...
The Internet of Everything (IoE) brings together people, process, data and things to make networked connections more relevant and valuable than ever before – transforming information into knowledge and knowledge into wisdom. IoE creates new capabilities, richer experiences, and unprecedented opportunities to improve business and government operations, decision making and mission support capabilities. In his session at @ThingsExpo, Gary Hall, Chief Technology Officer, Federal Defense at Cisco S...
SYS-CON Events announced today that Vitria Technology, Inc. will exhibit at SYS-CON’s @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Vitria will showcase the company’s new IoT Analytics Platform through live demonstrations at booth #330. Vitria’s IoT Analytics Platform, fully integrated and powered by an operational intelligence engine, enables customers to rapidly build and operationalize advanced analytics to deliver timely business outcomes ...
DevOps is about increasing efficiency, but nothing is more inefficient than building the same application twice. However, this is a routine occurrence with enterprise applications that need both a rich desktop web interface and strong mobile support. With recent technological advances from Isomorphic Software and others, it is now feasible to create a rich desktop and tuned mobile experience with a single codebase, without compromising performance or usability.
“We just completed the roll out of our first public and private cloud offerings, which are a combination of public, hybrid, and private cloud,” stated Erik Levitt, CEO of Open Data Centers, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the...
The Internet of Things (IoT) is causing data centers to become radically decentralized and atomized within a new paradigm known as “fog computing.” To support IoT applications, such as connected cars and smart grids, data centers' core functions will be decentralized out to the network's edges and endpoints (aka “fogs”). As this trend takes hold, Big Data analytics platforms will focus on high-volume log analysis (aka “logs”) and rely heavily on cognitive-computing algorithms (aka “cogs”) to mak...
SYS-CON Events announced today that GENBAND, a leading developer of real time communications software solutions, has been named “Silver Sponsor” of SYS-CON's WebRTC Summit, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. The GENBAND team will be on hand to demonstrate their newest product, Kandy. Kandy is a communications Platform-as-a-Service (PaaS) that enables companies to seamlessly integrate more human communications into their Web and mobile applicatio...
VictorOps is making on-call suck less with the only collaborative alert management platform on the market. With easy on-call scheduling management, a real-time incident timeline that gives you contextual relevance around your alerts and powerful reporting features that make post-mortems more effective, VictorOps helps your IT/DevOps team solve problems faster.
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
Companies today struggle to manage the types and volume of data their customers and employees generate and use every day. With billions of requests daily, operational consistency can be elusive. In his session at Big Data Expo, Dave McCrory, CTO at Basho Technologies, will explore how a distributed systems solution, such as NoSQL, can give organizations the consistency and availability necessary to succeed with on-demand data, offering high availability at massive scale.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, de...
The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information. In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focu...
SYS-CON Media announced that IBM, which offers the world’s deepest portfolio of technologies and expertise that are transforming the future of work, has launched ad campaigns on SYS-CON’s numerous online magazines such as Cloud Computing Journal, Virtualization Journal, SOA World Magazine, and IoT Journal. IBM’s campaigns focus on vendors in the technology marketplace, the future of testing, Big Data and analytics, and mobile platforms.
Security can create serious friction for DevOps processes. We've come up with an approach to alleviate the friction and provide security value to DevOps teams. In her session at DevOps Summit, Shannon Lietz, Senior Manager of DevSecOps at Intuit, will discuss how DevSecOps got started and how it has evolved. Shannon Lietz has over two decades of experience pursuing next generation security solutions. She is currently the DevSecOps Leader for Intuit where she is responsible for setting and driv...