Welcome!

Machine Learning Authors: Pat Romanski, Liz McMillan, Yeshim Deniz, Elizabeth White, Corey Roth

Related Topics: Java IoT, Industrial IoT, Weblogic, Eclipse, Machine Learning , Recurring Revenue

Java IoT: Article

Column Store, In-Memory, MPP Databases and Oracle

How Oracle implements latest developments in database world

( For latest information on Oracle 12c database update please refer to the following article: Oracle 12c Database and How It Relates to SAP Hana )

RDBMSs are stable and mature products. While there is nothing radically new on horizon that would challenge Codd's relational theory and related advances in data processing there are some developments that force established vendors like Oracle to come up with new features and products.

Column Stores and Oracle
Column store concept has been around for quite a while. Vendors like HP Vertica grabbed some market share in data warehousing segment with their column store,  MPP databases. Oracle Exadata is offering Hybrid Columnar Compression -  solution that modifies Oracle standard row based storage (NSM, or row-major) into proprietary format that is probably closer to PAX (as opposed to DSM) classification. Rows of data are reorganized, broken down into columns, compressed and stored in Compression Units which consist of multiple Oracle data blocks. CU is physically implemented as a standard Oracle single column chained row. (Description of CU layout is based on marvelous article written by Oracle expert  Jonathan Lewis).

This is truly hybrid design, i.e., column store is implemented on top of a standard row store.

It is not in the scope of this article to discuss pros and cons of this implementation from performance, locking, compression and other points of view. I will just mention that HCC requires change in standard operating procedures and methods.

MPP (shared nothing) and Oracle
Column-based stores like HP Vertica use Multiple Parallel Processing, shared nothing design to enhance performance by bringing processing closer to data, i.e., data is processed in parallel on the node where it resides. Volume of data that is moved around is reduced with the additional benefit of CPU and data proximity.

Oracle's implementation of this idea could be classified as asymmetric MPP. Oracle Exadata uses offloading to storage layer, Smart Scan, Storage Indexes and other techniques to improve performance.

Storage layer (Exadata cells) are tasked with as much work as possible to reduce load on database server and network. Each Exadata storage cell has the ability to perform some parts of data processing operations as well as decompression.

IMDB and Oracle
SAP relatively recently released Hana - a fully functional in-memory RDBMS, targeted for both OLTP and OLAP applications. Hana operates on the premise that whole database is in memory and not on disk. Majority of data processing is now pointer based arithmetic, so whole sections of RDBMS code related to moving data back and forth between disk, RAM and CPU are not needed any more. This is all possible because memory is more affordable and abundant, so much so that most of modern OLTP databases can completely fit within modern server's RAM.

Oracle puchased Times Ten in-memory database, but marketed it mostly as caching layer to standard Oracle database. Times Ten is not marketed as stand alone IMDB the way SAP Hana is.

Oracle database can have Flash Cache devices configured as an extension of SGA for better performance (via database parameter), or for database logging purposes.

Exadata can be configured with terabytes of Flash Cache memory for database caching and to serve as solid state disk. This is not memory directly accessible by CPU though ( DRAM ), i.e., Oracle database accesses Flash Cache via PCI interface and IO operating system calls. In other words, Flash is treated the same as disk, with all negative consequences of such approach regarding code complexity and performance. The latest release of Exadata performs writes directly to flash cache first to improve performance. We should expect more optimizations that will try to better utilize abundance of various types of memory. Expected scenario could be similar to Microsoft Hekaton project is also about adding IMDB features to SQL Server ( tables can be loaded in memory and processed in IMDB fashion, with reduced latching and locking; perhaps choice between different storage engines will be possible).

Conclusion
Oracle will probably continue to execute on strategy that worked well in the past - gradual inclusion of new technologies into its core RDBMS product (like it did with programmable server, OODBMS, Internet database, XML, partitioning, etc.). None of the latest developments in server technologies and database world is as seismic as introduction of RDBMS, client-server and Internet computing was. Oracle was so far successful  in modifying its database engine to adjust to the changes in hardware and data processing methods. We expect this trend to continue, as pace of technological innovation is somewhat slowing down and no truly disruptive changes are on horizon. Oracle solutions are designed to introduce and take advantage of these new (old) technologies and avoid cannibalizing existing profits. Intent is to maximally protect existing legacy RDBMS software revenues and integrate and sell products that came with new hardware and software acquisitions. Oracle Exadata, for example, could be viewed just as intelligent, Oracle database aware and Oracle produced SAN, bundled with database server. It is perhaps safe strategy in an environment where even mediocre and lackluster repackaging, modifying and integrating acquisitions is not seriously challenged by radical new ideas or strong, competitive implementations of existing concepts and technologies. SAP Hana, for example, is also unification layer built on top of in-house built or acquired products.

More Stories By Ranko Mosic

Ranko Mosic, BScEng, is specializing in Big Data/Data Architecture consulting services ( database/data architecture, machine learning ). His clients are in finance, retail, telecommunications industries. Ranko is welcoming inquiries about his availability for consulting engagements and can be reached at 408-757-0053 or [email protected]

@CloudExpo Stories
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | CloudEXPO New York event will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018.
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors!
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
Sanjeev Sharma Joins November 11-13, 2018 @DevOpsSummit at @CloudEXPO New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
As Cybric's Chief Technology Officer, Mike D. Kail is responsible for the strategic vision and technical direction of the platform. Prior to founding Cybric, Mike was Yahoo's CIO and SVP of Infrastructure, where he led the IT and Data Center functions for the company. He has more than 24 years of IT Operations experience with a focus on highly-scalable architectures.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.