Welcome!

AJAX & REA Authors: Piram Manickam, Subrahmanya SV, S Sangeetha, Bob Gourley, RealWire News Distribution

Blog Feed Post

The Structure of Big Data

First things first, all data is more or less structured. That being said, there is…

  • Structured Data
  • Semi-Structured Data
  • Unstructured Data

I tend to think of it as: data, composite or simple, with or without content. In that context, email is structured composite data (from, to, subject, date) with unstructured content (message body). The composite data is structured. The content is unstructured. Though simple data may or may not be structured. The ‘subject’ data is unstructured. The ‘to’ data is structured. It is composed of a local-part (username) and a domain.

While content is unstructured, there may be an implied structure.

So what is the difference between structured data, semi-structured data, and unstructured data?

Structured Data

The structure is externally enforced. Data

  • The data is stored in a database.
    • Relational
      • Transaction
    • XML
      • Catalog

The data itself is not structured. The structure is defined by the database. The data would be semi-structured if it was exported and transformed into JSON or XML.

Semi-Structured Data

The structure is self defined. Data and / or Content

  • The data is stored as text.
    • JSON
      • User Profile
    • XML
      • Application / Form

Unstructured Data

The structure of the data is externally defined. Metadata and Content

  • The data is stored in a binary format and / or document.
    • Media (e.g. Ogg).
    • Video (e.g. Vorbis).
    • Audio (e.g. Theora).
    • Image (e.g. PNG).
    • Microsoft Word
    • Adobe PDF

The structure is defined by the file format. The data is composed of structured metadata and unstructured content. That being said, a video is composed of frames and an image is composed of pixels.

  • The data is stored as plain text.
    • Log

The structure is defined by a pattern in the logging configuration file. The data is composed of structured metadata (e.g. severity) and unstructured content (log message).

  • The data is user generated.
    • Status (Facebook)
    • Tweet (Twitter)
    • Comment (WordPress)

The structure is defined by the application / form. The data is composed of structured metadata (e.g. user ID) and unstructured content (user message).

Update

I would say that the structure of content is user defined and thus interpreted. However, content is often a component of data (unstructured, externally defined). Though if I typed up this post in gedit and saved it as a text file, that might constitute content independent of data.


With all all that said, the structure of data is not exactly black and white.


Read the original blog entry...

More Stories By Daniel Thompson

I curate the content on this page, but the credit goes to my talented colleagues for the posts that you see here. Much of what you read on this page is the work of friends at How to JBoss, and I encourage you to drop by the site at http://www.howtojboss.com for some of the best JBoss technical and non-technical content for developers, architects and technology executives on the Web.

Cloud Expo Breaking News
Enterprises can't close their doors just because integration tools won't cope with the volume of information that their systems produce. As each day goes by, their information will become larger and more complicated, and enterprises must constantly struggle to manage the integration of dozens (or hundreds) of systems. Apache Hadoop has quickly become the technology of choice for enterprises that need to perform complex analysis of petabytes of data, but few are aware of its potential to hand...
Companies around the world are collecting massive amounts of data everyday that’s sitting around and not being utilized. Take for example the fact that companies collect demographic and location-based data via mobile devices all the time, but have to figure out how to monetize that data. In this session, Joyent CTO and founder Jason Hoffman will examine the state of Big Data, taking a look at what we're doing now to discussing what's on the horizon, as companies prepare and realign their busines...
Planning scalable environments isn't terribly difficult, but it does require a change of perspective. During this session we'll broaden our views to think on an Internet Scale by dissecting a video publishing application built with The SoftLayer Platform, Message Queuing, Object Storage, and Drupal. By examining a scalable modular application build that can handle unpredictable traffic, you'll be able to grow your development arsenal and pick up a few strategies to apply to your own projects.
Learn about the complex regulations surrounding HIPAA compliance and other considerations for running sensitive data in the Cloud. In their session at the 12th International Cloud Expo, Ken Ziegler, CEO of Logicworks, and Frank Nydam, Director of Healthcare Solutions at VMware, will discuss the best practices for leveraging virtualization and cloud technologies without sacrificing security or compliance. Care providers, State and Federal entities, integrators and SaaS providers large and small...
Backup, Recovery, and Archiving (BURA) are critical elements for IT to address. BURA solutions need to address a broad spectrum of needs including data protection, regulatory compliance, and business continuity. Today's cloud based solutions can enable customers to procure and consume BURA as a service supported by EMC's latest technologies. At the end of Rich Place's session at 12th Cloud Expo | Cloud Expo New York [June 10-13, 2013], you will be able to: 1. Gain a full understanding of Backup,...
A recent study by analyst firm IDC reports that in 2012, 1.7 million cloud computing-related roles across the globe could not be filled due to the lack of training, certification and experience in the applicant pool. As the global demand for cloud and big data expertise increases, employers are finding it difficult to recruit talent, which is slowing down the ability for organizations to adopt, implement, and realize benefits from innovative platforms like OpenStack. In this session join Clo...
Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how. In this session Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the importance of knowing who SMB...
Compelling consumer applications are created every day. Are you ready for the IT implications both internally and externally? As your datacenter needs more capacity, the cloud will be critical to success. What are the key considerations to help plan for the needed capacity over time? And how can the cloud best work with your existing applications? In his General Session at the 12th International Cloud Expo, Brian Jawalka, Enterprise Solutions Architect at Rackspace Hosting, will open conversat...
Cloud computing is more than a buzz-phrase it’s a transformative IT paradigm shift. The emphasis in the cloud is on elasticity, scalability, agility and open. Not just open standards but open APIs and open source. The delivery of software is also going through a paradigm shift. Open source software was often a commoditization of a market leader; Unix to Linux or Oracle to MySQL what’s changing is that the iterative nature, user context and the motto of releasing early and often are driving real ...
These days, it seems that every cloud provider claims that cloud is safer than your traditional datacenter. Is it though? In his General Session at 12th Cloud Expo | Cloud Expo New York, McAfee expert Rishi Bhargava will help you explore and address the security challenges and considerations for public cloud (IaaS, PaaS and SaaS).