Welcome!

Machine Learning Authors: William Schmarzo, Carmen Gonzalez, Pat Romanski, Liz McMillan, Yeshim Deniz

Related Topics: Java IoT, Industrial IoT, Microservices Expo, Eclipse, Machine Learning , Apache

Java IoT: Article

The Disruptor Framework: A Concurrency Framework for Java

Rediscovering the Producer-Consumer Model with the Disruptor

Let's start with the basic question: What is the disruptor? The disruptor is a concurrency framework for Java that allows data sharing between threads. The age old way of coding a producer-consumer model is to use a queue as the buffer area between the producer and the consumer, where the producer adds data objects to the queue, which are in turn processed by the consumer. However, such a model does not work well at the hardware level and ends up being highly inefficient. The disruptor in its simplest form replaces the queue with a data structure known as the ‘ring buffer'. Which brings us to the next question, what is the ring buffer? The ring buffer is an array of fixed length (which must be a power of 2), it's circular and wraps. This data structure is at the core of what makes the disruptor super fast.

Let's explore a simple everyday scenario in enterprise architectures. A producer (let's call it the publisher) creates data and stores it in the queue. Two immediate consumers (let's call them fooHandler and barHandler) consume the data and make updates to it. Once these 2 processors are done with a piece of data, it is then passed on to a third consumer (let's call it fooBarHandler) for further processing. In a concurrent processing system using legacy techniques, coding this architecture would involve a crisscross of queues and numerous concurrency challenges, such as dealing with locks, CAS, write contention, etc. The disruptor on the other hand immensely simplifies such a scenario by providing a simple API for creating the producer, consumers and ring buffer, which in turn relieve the developer of all concerns surrounding handling concurrency and doing so in an efficient manner. We shall now explore how the disruptor works its magic and provides a reliable messaging framework.

Writing to the ring buffer

Looking at the figure above, we find ourselves in the middle of the action. The ring buffer is an array of length 4 and is populated with data items - 4,5,6 and 7, which in the case of the disruptor are known as events. The square above the ring buffer containing the number 7 is the current sequence number, which denotes the highest populated event in the ring buffer. The ring buffer keeps track of this sequence number and increments it as and when new events are published to it. The fooHandler, barHandler and fooBarHandler are the consumers, which in disruptor terminology are called ‘event processors'. Each of these also has a square containing a sequence number, which in the case of the event processors denotes the highest event that they have consumed/processed so far. Thus its apparent that each entity (except the publisher) tracks its own sequence number and thus does not need to rely on a third party to figure out which is the next event its after.

The publisher asks the ring buffer for the next sequence number. The ring buffer is currently at 7, so the next sequence number would be 8. However, this would also entail overwriting the event with sequence number 4 (since there are only 4 slots in the array and the oldest event gets replaced with the newest one). The ring buffer first checks the most downstream consumer (fooBarHandler) to determine whether it is done processing the event with sequence number 4. In this case, it has, so it returns the number 8 to the publisher. In case fooBarHandler was stuck at a sequence number lower than 4, the ring buffer would have waited for it to finish processing the 4th event before returning the next sequence number to the publisher. This sequence number helps the publisher identify the next available slot in the ring buffer by performing a simple mod operation. indexOfNextAvailableSlot = highestSeqNo%longthOfRingBuffer, which in this case is 0 (8%4). The publisher then claims the next slot in the ring buffer (via a customizable strategy depending on whether there is a single or multiple publishers), which is currently occupied by event 4, and publishes event 8 to it.

Reading from the ring buffer by immediate consumers

The figure above shows the state of operations after the publisher has published event 8 to the ring buffer. The ring buffer's sequence number has been updated to 8 and now contains events 5,6,7 and 8. We see that foohandler, which has processed events upto 7, has been waiting (using a customizable strategy) for the 8th event to be published. Unlike the publisher though, it does not directly communicate with the ring buffer, but uses an entity known as the ‘sequence barrier' to do so on its behalf. The sequence barrier let's fooHandler know that the highest sequence number available in the ring buffer is now 8. FooHandler may now get this event and process it.

Similarly, barHandler checks the sequence barrier to determine whether there are any more events it can process. However, rather than just telling barHandler that the next (6th) event is up for grabs, the sequence barrier returns the highest sequence number present in the ring buffer to barHandler too. This way, barHandler can grab events 6,7,8 and process them in a batch before it has to enquire about further events being published. This saves time and reduces load.

Another important thing to note here is that in the case of multiple event processors, any given field in the event object must only be written to by any one event processor. Doing so prevents write contention, and thus removes the need for locks or CAS.

Reading from the ring buffer by downstream consumers

A few moments after the set of immediate consumers grab the next set of data, the state of affairs looks like the figure above. fooHandler is done processing all 8 available events (and has accordingly updated its sequence number to 8), whereas barHandler, being the slow coach that it is, has only processed events upto number 6 (and thus has updated sequence number to 6). We now see that fooBarHandler, which was done processing events upto number 5 at the start of our examination, is still waiting for an event higher than that to process. Why did its sequence barrier not inform it once event 8 was published to the ring buffer? Well, that is because downstream consumers don't automatically get notified of the highest sequence number present in the ring buffer. Their sequence barriers on the other hand determine the next sequence number they can process by calculating the minimum sequence number that the set of event processors directly before them have processed. This helps ensure that the downstream consumers only act on an event once its processing has been completed by the entire set of upstream consumers. The sequence barrier examines the sequence number on fooHandler (which is 8) and the sequence number on barHandler (which is 6) and decides that event 6 is the highest event that fooBarHandler can process. It returns this info to fooBarHandler, which then grabs event 6 and processes it. It must be noted that even in the case of the downstream consumers, they grab the events directly from the ring buffer and not from the consumers before them.

Well, that is about all you would need to know about the working of the disruptor framework to get started. But while this is all well and good in theory, the question still remains, how would one code the above architecture using the disruptor library? The answer to that question lies below.

Coding the disruptor

public final class FooBarEvent {
private double foo=0;
private double bar=0;
public double getFoo(){
return foo;
}
public double getBar() {
return bar;
}
public void setFoo(final double foo) {
this.foo = foo;
}
public void setBar(final double bar) {
this.bar = bar;
}
public final static EventFactory<FooBarEvent> EVENT_FACTORY
= new EventFactory<FooBarEvent>() {
public FooBarEvent newInstance() {
return new FooBarEvent();
}
};
}

The class FooBarEvent, as the name suggests, acts as the event object which is published by the publisher to the ring buffer and consumed by the eventProcessors - fooHandler, barHandler and fooBarHandler. It contains two fields ‘foo' and ‘bar' of type double, along with their corresponding setters/getters. It also contains an entity ‘EVENT_FACTORY' of type EventFactory, which is used to create an instance of this event.

public class FooBarDisruptor {           
public static final int RING_SIZE=4;
public static final ExecutorService EXECUTOR
=Executors.newCachedThreadPool();

final EventTranslator<FooBarEvent> eventTranslator
=new EventTranslator<FooBarEvent>() {
public void translateTo(FooBarEvent event,
long sequence) {
double foo=event.getFoo();
double bar=event.getBar();
system.out.println("foo="+foo
+", bar="+bar
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> fooHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double foo=Math.random();
event.setFoo(foo);
System.out.println("setting foo to "+foo
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> barHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double bar=Math.random();
event.setBar(bar);
System.out.println("setting bar to "+bar
+" (sequence="+sequence+")");
}
};

final EventHandler<FooBarEvent> fooBarHandler
= new EventHandler<FooBarEvent>() {
public void onEvent(final FooBarEvent event,
final long sequence,
final boolean endOfBatch)
throws Exception {
double foo=event.getFoo();
double bar=event.getBar();
System.out.println("foo="+foo
+", bar="+bar
+" (sequence="+sequence+")");
}
};

public Disruptor setup() {
Disruptor<FooBarEvent> disruptor =
new Disruptor<FooBarEvent>(FooBarEvent.EVENT_FACTORY,
EXECUTOR,
new SingleThreadedClaimStrategy(RING_SIZE),
new SleepingWaitStrategy());
disruptor.handleEventsWith(fooHandler, barHandler).then(fooBarHandler);
RingBuffer<FooBarEvent> ringBuffer = disruptor.start();             
return disruptor;
}

public void publish(Disruptor<FooBarEvent> disruptor) {
for(int i=0;i<1000;i++) {
disruptor.publishEvent(eventTranslator);
}
}

public static void main(String[] args) {
FooBarDisruptor fooBarDisruptor=new FooBarDisruptor();
Disruptor disruptor=fooBarDisruptor.setup();
fooBarDisruptor.publish(disruptor);
}
}

The class FooBarDisruptor is where all the action happens. The ‘eventTranslator' is an entity which aids the publisher in publishing events to the ring buffer. It implements a method ‘translateTo' which gets invoked when the publisher is granted permission to publish the next event. fooHandler, barHandler and fooBarHandler are the event processors, and are objects of type ‘EventHandler'. Each of them implements a method ‘onEvent' which gets invoked once the event processor is granted access to a new event. The method ‘setup' is responsible for creating the disruptor, assigning the corresponding event handlers, and setting the dependency rules amongst them. The method ‘publish' is responsible for publishing a thousand events of the type ‘FooBarEvent' to the ring buffer.

In order to get the above code to work, you must download the disruptor jar file from http://code.google.com/p/disruptor/downloads/list and include the same in your classpath.

Conclusion
The disruptor is currently in use in the ultra efficient LMAX architecture, where it has proven to be a reliable model for inter thread communication and data sharing, reducing the end to end latency to a fraction of what queue based architectures provided. It does so using a variety of techniques, including replacing the array blocking queue with a ring buffer, getting rid of all locks, write contention and CAS operations (except in the scenario where one has multiple publishers), having each entity track its own progress by way of a sequence number, etc. Adopting this framework can greatly boost a developer's productivity in terms of coding a producer-consumer pattern, while at the same time aid in creating an end product far superior in terms of both design and performance to the legacy queue based architectures.

More Stories By Sanat Vij

Sanat Vij is a professional software engineer currently working at CenturyLink. He has vast experience in developing high availability applications, configuring application servers, JVM profiling and memory management. He specializes in performance tuning of applications, reducing response times, and increasing stability.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
Cloud promises the agility required by today’s digital businesses. As organizations adopt cloud based infrastructures and services, their IT resources become increasingly dynamic and hybrid in nature. Managing these require modern IT operations and tools. In his session at 20th Cloud Expo, Raj Sundaram, Senior Principal Product Manager at CA Technologies, will discuss how to modernize your IT operations in order to proactively manage your hybrid cloud and IT environments. He will be sharing be...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
Quickly find the root cause of complex database problems slowing down your applications. Up to 88% of all application performance issues are related to the database. DPA’s unique response time analysis shows you exactly what needs fixing - in four clicks or less. Optimize performance anywhere. Database Performance Analyzer monitors on-premises, on VMware®, and in the Cloud, including Amazon® AWS and Azure™ virtual machines.
NHK, Japan Broadcasting, will feature the upcoming @ThingsExpo Silicon Valley in a special 'Internet of Things' and smart technology documentary that will be filmed on the expo floor between November 3 to 5, 2015, in Santa Clara. NHK is the sole public TV network in Japan equivalent to the BBC in the UK and the largest in Asia with many award-winning science and technology programs. Japanese TV is producing a documentary about IoT and Smart technology and will be covering @ThingsExpo Silicon Val...
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
NHK, Japan Broadcasting, will feature the upcoming @ThingsExpo Silicon Valley in a special 'Internet of Things' and smart technology documentary that will be filmed on the expo floor between November 3 to 5, 2015, in Santa Clara. NHK is the sole public TV network in Japan equivalent to the BBC in the UK and the largest in Asia with many award-winning science and technology programs. Japanese TV is producing a documentary about IoT and Smart technology and will be covering @ThingsExpo Silicon Val...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in compute, storage and networking technologies, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/...
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Analytic. Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
Building a cross-cloud operational model can be a daunting task. Per-cloud silos are not the answer, but neither is a fully generic abstraction plane that strips out capabilities unique to a particular provider. In his session at 20th Cloud Expo, Chris Wolf, VP & Chief Technology Officer, Global Field & Industry at VMware, will discuss how successful organizations approach cloud operations and management, with insights into where operations should be centralized and when it’s best to decentraliz...
Data is an unusual currency; it is not restricted by the same transactional limitations as money or people. In fact, the more that you leverage your data across multiple business use cases, the more valuable it becomes to the organization. And the same can be said about the organization’s analytics. In his session at 19th Cloud Expo, Bill Schmarzo, CTO for the Big Data Practice at Dell EMC, introduced a methodology for capturing, enriching and sharing data (and analytics) across the organization...
Developers want to create better apps faster. Static clouds are giving way to scalable systems, with dynamic resource allocation and application monitoring. You won't hear that chant from users on any picket line, but helping developers to create better apps faster is the mission of Lee Atchison, principal cloud architect and advocate at New Relic Inc., based in San Francisco. His singular job is to understand and drive the industry in the areas of cloud architecture, microservices, scalability ...
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
Bert Loomis was a visionary. This general session will highlight how Bert Loomis and people like him inspire us to build great things with small inventions. In their general session at 19th Cloud Expo, Harold Hannon, Architect at IBM Bluemix, and Michael O'Neill, Strategic Business Development at Nvidia, discussed the accelerating pace of AI development and how IBM Cloud and NVIDIA are partnering to bring AI capabilities to "every day," on-demand. They also reviewed two "free infrastructure" pr...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...