Welcome!

IoT User Interface Authors: Elizabeth White, John Basso, Kevin Jackson, Richard Hale, Liz McMillan

Related Topics: Microservices Expo, Java IoT, Microsoft Cloud, IoT User Interface

Microservices Expo: Article

From Data to Information to Knowledge to Intelligence

What is the minimum metadata that is absolutely key to understanding a little bit how this data came about?

The debate between data and information has been going on for quite some time. When people say "knowledge is power", are they referring to data or information? Is knowledge different still? And how about "intelligence" where does that fit?

How can we go from data to information to knowledge to intelligence?

The answer is simple. By understanding the animated nature of data evolution and transformation, and acting upon this understanding.

And this is brought to light by logs from your Information Systems.

Understand this and unleash the Power of Logs.

Figure 1 - Data to Information to Knowledge to Intelligence, and the role of logs as metadata

Data seems mainly one-dimensional.

Consult any data base or data warehouse, perform even complex queries on these and you will get a "flat" answer.

The fact that you get a single answer will make you think that data is absolute, where in fact it's not.

In fact, data is relative. And it has a very high frequency of change!

For example data is relative in time, and today's flat answer may not be yesterday's flat answer and probably not tomorrow's flat answer.

Figure 2 describes the different iterations of flat data, and how these snapshots and iterations tie together to form the movie of the ever-evolving nature of data.

Figure 2 - Flat, one-dimensional data and multi-dimensional time-based data evolution

And understanding the ever-changing and evolving nature of data is key to transforming data into information. Indeed one has to understand the cycles of change of data in order to pretend to gain information.

Our brain is a superior machine for this transformation as we are able to subconsciously and automatically model, infer, expand, project and otherwise manipulate data as well as recognize patterns, differences, similarities and other properties of these data, allowing us a higher level of understanding and abstraction.

This is how our brain transforms huge amounts of data into information, by processing it, by transforming it.

The data about how to transform data, data about data, is called metadata.

Data is the raw material, the metadata is the way to manipulate the data, and information is the result of this manipulation, much like the fruit is the raw material and the juice is the result of its manipulation.

Figure 3 - Metadata as key component to understand data evolution

We are dealing here with very complex concepts and mechanisms in information technology. This is a fairly new science; one of the oldest theoretical studies of data was Shannon's entropy theorem. Shannon studied how much information was and could be contained in data.

Again, data about data, metadata.

Metadata is key to transforming data into information.

Figure 4 - Detail on metadata as key component from one iteration to another of flat data

Although our brains are expert systems in managing metadata and we are using heuristics so effective that we touch the very own nature of the magic of human kind, when transposed to Information Technology we have very clear and very practical limitations.

For example, it is simply not practical to keep, along with the data, all the previous versions of that data along with the reasons why this data had to change and generally all the context surrounding that change, and this for every iteration and version of the data. This is impractical because it would imply:

  1. An immense storage space
  2. An Artificial Intelligence system that is expert at sorting through the universe of metadata and extracting the one used to reconstruct in real time the lifecycle of the data

Too bad, as this is really what could eliminate data flatness and put back into perspective the animated nature of data in all its glory.

Okay, so what is it that we could do then? What is the minimum metadata that is absolutely key to understanding a little bit how this data came about?

Stay tuned for next installment!!

More Stories By Gorka Sadowski

Gorka is a natural born entrepreneur with a deep understanding of Technology, IT Security and how these create value in the Marketplace. He is today offering innovative European startups the opportunity to benefit from the Silicon Valley ecosystem accelerators. Gorka spent the last 20 years initiating, building and growing businesses that provide technology solutions to the Industry. From General Manager Spain, Italy and Portugal for LogLogic, defining Next Generation Log Management and Security Forensics, to Director Unisys France, bringing Cloud Security service offerings to the market, from Director of Emerging Technologies at NetScreen, defining Next Generation Firewall, to Director of Performance Engineering at INS, removing WAN and Internet bottlenecks, Gorka has always been involved in innovative Technology and IT Security solutions, creating successful Business Units within established Groups and helping launch breakthrough startups such as KOLA Kids OnLine America, a social network for safe computing for children, SourceFire, a leading network security solution provider, or Ibixis, a boutique European business accelerator.

@CloudExpo Stories
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor – all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
Predictive analytics tools monitor, report, and troubleshoot in order to make proactive decisions about the health, performance, and utilization of storage. Most enterprises combine cloud and on-premise storage, resulting in blended environments of physical, virtual, cloud, and other platforms, which justifies more sophisticated storage analytics. In his session at 18th Cloud Expo, Peter McCallum, Vice President of Datacenter Solutions at FalconStor, discussed using predictive analytics to mon...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
SYS-CON Events announced today that Isomorphic Software will exhibit at DevOps Summit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Isomorphic Software provides the SmartClient HTML5/AJAX platform, the most advanced technology for building rich, cutting-edge enterprise web applications for desktop and mobile. SmartClient combines the productivity and performance of traditional desktop software with the simp...
"When you think about the data center today, there's constant evolution, The evolution of the data center and the needs of the consumer of technology change, and they change constantly," stated Matt Kalmenson, VP of Sales, Service and Cloud Providers at Veeam Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
We all know the latest numbers: Gartner, Inc. forecasts that 6.4 billion connected things will be in use worldwide in 2016, up 30 percent from last year, and will reach 20.8 billion by 2020. We're rapidly approaching a data production of 40 zettabytes a day – more than we can every physically store, and exabytes and yottabytes are just around the corner. For many that’s a good sign, as data has been proven to equal money – IF it’s ingested, integrated, and analyzed fast enough. Without real-ti...
Actian Corporation has announced the latest version of the Actian Vector in Hadoop (VectorH) database, generally available at the end of July. VectorH is based on the same query engine that powers Actian Vector, which recently doubled the TPC-H benchmark record for non-clustered systems at the 3000GB scale factor (see tpc.org/3323). The ability to easily ingest information from different data sources and rapidly develop queries to make better business decisions is becoming increasingly importan...
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Cloud analytics is dramatically altering business intelligence. Some businesses will capitalize on these promising new technologies and gain key insights that’ll help them gain competitive advantage. And others won’t. Whether you’re a business leader, an IT manager, or an analyst, we want to help you and the people you need to influence with a free copy of “Cloud Analytics for Dummies,” the essential guide to this explosive new space for business intelligence.
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Qosmos has announced new milestones in the detection of encrypted traffic and in protocol signature coverage. Qosmos latest software can accurately classify traffic encrypted with SSL/TLS (e.g., Google, Facebook, WhatsApp), P2P traffic (e.g., BitTorrent, MuTorrent, Vuze), and Skype, while preserving the privacy of communication content. These new classification techniques mean that traffic optimization, policy enforcement, and user experience are largely unaffected by encryption. In respect wit...
ReadyTalk has expanded the capabilities of the FoxDen collaboration platform announced late last year to include FoxDen Connect, an in-room video collaboration experience that launches with a single touch. With FoxDen Connect, users can now not only engage in HD video conferencing between iOS and Android mobile devices or Chrome browsers, but also set up in-person meeting rooms for video interactions. A host’s mobile device automatically recognizes the presence of a meeting room via beacon tech...
Manufacturers are embracing the Industrial Internet the same way consumers are leveraging Fitbits – to improve overall health and wellness. Both can provide consistent measurement, visibility, and suggest performance improvements customized to help reach goals. Fitbit users can view real-time data and make adjustments to increase their activity. In his session at @ThingsExpo, Mark Bernardo Professional Services Leader, Americas, at GE Digital, discussed how leveraging the Industrial Internet a...
On Dice.com, the number of job postings asking for skill in Amazon Web Services increased 76 percent between June 2015 and June 2016. Salesforce.com saw its own skill mentions increase 37 percent, while DevOps and Cloud rose 35 percent and 28 percent, respectively. Even as they expand their presence in the cloud, companies are also looking for tech professionals who can manage projects, crunch data, and figure out how to make systems run more autonomously. Mentions of ‘data science’ as a skill ...
In his session at Cloud Expo, Robert Cohen, an economist and senior fellow at the Economic Strategy Institute, will provide economic scenarios that describe how the rapid adoption of software-defined everything including cloud services, SDDC and open networking will change GDP, industry growth, productivity and jobs. This session will also include a drill down for several industries such as finance, social media, cloud service providers and pharmaceuticals.
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...
Choosing the right cloud for your workloads is a balancing act that can cost your organization time, money and aggravation - unless you get it right the first time. Economics, speed, performance, accessibility, administrative needs and security all play a vital role in dictating your approach to the cloud. Without knowing the right questions to ask, you could wind up paying for capacity you'll never need or underestimating the resources required to run your applications.
Security, data privacy, reliability and regulatory compliance are critical factors when evaluating whether to move business applications from in-house client hosted environments to a cloud platform. In her session at 18th Cloud Expo, Vandana Viswanathan, Associate Director at Cognizant, In this session, will provide an orientation to the five stages required to implement a cloud hosted solution validation strategy.
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.