|By Hovhannes Avoyan||
|August 9, 2012 07:00 AM EDT||
As systems administrators, application logs are often the key to our success, but also our biggest hassle. They provide clues to what’s going on when things go awry, and in those situations more detail is generally better. But when you don’t actually know something is wrong, and just want to get a sense for whether things are normal, more detail can create so much noise that it’s all but impossible to glean any useful information.
In those situations, you’d rather just have statistical information about what’s in your logs. In this article, I present a simple and easy solution to turn your logs into useful graphs, in real time. If you ever need to measure the volume of your logs, or perhaps graph the frequency of certain log events, then read on.
The solution I present uses four key tools:
- Log4J (though plain log files would fit as well)
With so many moving parts, you might be tempted to think this could be an overcomplicated solution. But in fact — as in the long tradition of Unix command line tools — it is a composition of simple tools each doing one job very well. As with files piped from one Unix command to another, these four components act as a pipeline for log events, with each piece adding value to the stream along the way.
All of the log events in this article start inside of Log4J. If you run Java applications, then this provides an easy way to hook into your logs, to peel off an event stream that you want to see graphed in Monitis. But, Log4J could easily be replaced in this solution with plain log files, syslog, or any number of other logging frameworks.
The key modification that we make to Log4J is to add a SocketAppender that sends a copy of selected Loggers to our logstash server.
The role of logstash in the pipeline is twofold. First, it listens for connections from Java application servers, accepting streams of logs when they connect. Second, it filters, modifies, and routes those streams to the appropriate outputs. In this case, we’ll be handling all of the incoming streams by notifying StatsD each time a log event is received, without actually sending the content of each event.
Logstash will be receiving log events very frequently, but Monitis only wants to receive updates at most once per minute. To resolve this mismatch, StatsD acts as our log stream bean counter, allowing logstash to send increment messages each time an event is received. StatsD records these in counters for each type of log message, and then sends the counts on to Monitis every 60 seconds.
Finally, we get to the end of the pipeline, and Monitis receives the count messages. These are added to the appropriate custom monitors, which are automatically created if they don’t already exist. Once the data is in Monitis, it can be graphed in the Web UI, or used to send alerts when a rate of log events is outside of a user-specified threshold.
The gory details
Now that you’ve seen the overview, let’s take a look at the configuration details that make it happen. Don’t worry, since each component in the pipeline is doing a simple job, there’s really not much to it.
Install and configure the software
Let’s look at installation details for the tools in each step in the pipeline. I’m assuming that you already have Java applications using Log4J. If not, modifying the pipeline to read from log files, receive from syslog, or other options is pretty straightforward, but outside the scope of this article. For that, refer to the logstash documentation on how to set up other kinds of logstash inputs.
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
Sep. 26, 2016 03:15 AM EDT Reads: 1,072
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs a...
Sep. 26, 2016 03:00 AM EDT Reads: 1,750
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service. In his session at 19th Cloud Exp...
Sep. 26, 2016 03:00 AM EDT Reads: 924
SYS-CON Events announced today that Secure Channels will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. The bedrock of Secure Channels Technology is a uniquely modified and enhanced process based on superencipherment. Superencipherment is the process of encrypting an already encrypted message one or more times, either using the same or a different algorithm.
Sep. 26, 2016 02:45 AM EDT Reads: 1,546
While DevOps promises a better and tighter integration among an organization’s development and operation teams and transforms an application life cycle into a continual deployment, Chef and Azure together provides a speedy, cost-effective and highly scalable vehicle for realizing the business values of this transformation. In his session at @DevOpsSummit at 19th Cloud Expo, Yung Chou, a Technology Evangelist at Microsoft, will present a unique opportunity to witness how Chef and Azure work tog...
Sep. 26, 2016 01:45 AM EDT Reads: 1,661
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
Sep. 26, 2016 01:00 AM EDT Reads: 1,613
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
Sep. 26, 2016 01:00 AM EDT Reads: 1,954
I'm a lonely sensor. I spend all day telling the world how I'm feeling, but none of the other sensors seem to care. I want to be connected. I want to build relationships with other sensors to be more useful for my human. I want my human to understand that when my friends next door are too hot for a while, I'll soon be flaming. And when all my friends go outside without me, I may be left behind. Don't just log my data; use the relationship graph. In his session at @ThingsExpo, Ryan Boyd, Engi...
Sep. 26, 2016 12:30 AM EDT Reads: 1,245
IoT is fundamentally transforming the auto industry, turning the vehicle into a hub for connected services, including safety, infotainment and usage-based insurance. Auto manufacturers – and businesses across all verticals – have built an entire ecosystem around the Connected Car, creating new customer touch points and revenue streams. In his session at @ThingsExpo, Macario Namie, Head of IoT Strategy at Cisco Jasper, will share real-world examples of how IoT transforms the car from a static p...
Sep. 26, 2016 12:30 AM EDT Reads: 1,537
SYS-CON Events announced today that Pulzze Systems will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems, Inc. provides infrastructure products for the Internet of Things to enable any connected device and system to carry out matched operations without programming. For more information, visit http://www.pulzzesystems.com.
Sep. 26, 2016 12:15 AM EDT Reads: 1,820
If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
Sep. 25, 2016 10:45 PM EDT Reads: 1,584
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
Sep. 25, 2016 10:45 PM EDT Reads: 4,411
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...
Sep. 25, 2016 10:00 PM EDT Reads: 1,700
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management solutions, helping companies worldwide activate their data to drive more value and business insight and to transform moder...
Sep. 25, 2016 09:30 PM EDT Reads: 2,505
The Transparent Cloud-computing Consortium (abbreviation: T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data processing High speed and high quality networks, and dramatic improvements in computer processing capabilities, have greatly changed the nature of applications and made the storing and processing of data on the network commonplace.
Sep. 25, 2016 09:00 PM EDT Reads: 933
SYS-CON Events announced today that Bsquare has been named “Silver Sponsor” of SYS-CON's @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. For more than two decades, Bsquare has helped its customers extract business value from a broad array of physical assets by making them intelligent, connecting them, and using the data they generate to optimize business processes.
Sep. 25, 2016 07:30 PM EDT Reads: 2,619
Fact is, enterprises have significant legacy voice infrastructure that’s costly to replace with pure IP solutions. How can we bring this analog infrastructure into our shiny new cloud applications? There are proven methods to bind both legacy voice applications and traditional PSTN audio into cloud-based applications and services at a carrier scale. Some of the most successful implementations leverage WebRTC, WebSockets, SIP and other open source technologies. In his session at @ThingsExpo, Da...
Sep. 25, 2016 05:30 PM EDT Reads: 1,564
Traditional on-premises data centers have long been the domain of modern data platforms like Apache Hadoop, meaning companies who build their business on public cloud were challenged to run Big Data processing and analytics at scale. But recent advancements in Hadoop performance, security, and most importantly cloud-native integrations, are giving organizations the ability to truly gain value from all their data. In his session at 19th Cloud Expo, David Tishgart, Director of Product Marketing ...
Sep. 25, 2016 05:00 PM EDT Reads: 1,778
The vision of a connected smart home is becoming reality with the application of integrated wireless technologies in devices and appliances. The use of standardized and TCP/IP networked wireless technologies in line-powered and battery operated sensors and controls has led to the adoption of radios in the 2.4GHz band, including Wi-Fi, BT/BLE and 802.15.4 applied ZigBee and Thread. This is driving the need for robust wireless coexistence for multiple radios to ensure throughput performance and th...
Sep. 25, 2016 02:30 PM EDT Reads: 1,531
SYS-CON Events announced today the Kubernetes and Google Container Engine Workshop, being held November 3, 2016, in conjunction with @DevOpsSummit at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA. This workshop led by Sebastian Scheele introduces participants to Kubernetes and Google Container Engine (GKE). Through a combination of instructor-led presentations, demonstrations, and hands-on labs, students learn the key concepts and practices for deploying and maintainin...
Sep. 25, 2016 02:00 PM EDT Reads: 2,633