03-07-2013, 02:35 PM
Building the Operational Data Store on DB2 UDB Using IBM Data Replication, WebSphere MQ Family, and DB2 Warehouse Manager
Building the Operational.pdf (Size: 4.85 MB / Downloads: 330)
Introduction
Throughout history, successful organizations have managed to consistently
evolve with the changing conditions of the business environment. The
environment is changing at an exponential rate, causing organizations to
continually invent new and innovative ideas at an even more rapid pace. It is a
necessity that organizations adapt, evolve, and innovate to stay competitive.
Over the past few years, organizations have routinely used Business Intelligence
and Data Warehousing techniques to gather data together into a single
integrated view, to store historical data to and to make strategic business
decisions by analyzing this data. This used to be a background process.
However, organizations have become more confident in using this type of
information. They have seen increasing need to integrate data across business
processes, channels, and organizational departments. In the new economy,
customers are demanding immediate access to a fully integrated portfolio of
products and services. Organizations must quickly satisfy these demands by
changing to a more customer-centric infrastructure. In addition, they can see the
benefit of doing this in real-time.
Introducing the Operational Data Store
It is addressing business needs that has driven the development of the
Operational Data Store (ODS), a data structure designed around a specific set of
data — that is, integrated for a specific purpose. The ODS typically contains
current data which is dynamically updated in a real-time (or near-real-time)
manner. In short, an ODS is an architectural construct containing subject
oriented, integrated, volatile, current (or near-current), and detailed operational
information.
One of the questions organizations face in deploying an ODS is whether to use
their existing DW system to store the real-time updates for the ODS. While this
may work in some situations, most clients find that conflicting Service Level
Agreements (SLAs) make it more desirable to have the ODS and data
warehouse (DW) operate on distinct schemas, or even totally separate
environments. Examples of conflicting SLAs include the need to have data in the
ODS faster than it can be transformed into the standardized DW format, as well
as unresolvable business priority issues between the ODS user and the
demands placed on the DW by ad-hoc queries.
Business questions
The emergence of the Internet and e-commerce have created some unique and
interesting opportunities and challenges. We now have the technology to transfer
information to anyone anywhere in the world. With this new capability, customers
are asking for more and more customized products and services. This has
created competitive pressures to get these products and services to the market
in shorter and shorter time frames.
Many businesses have undergone a restructuring and reorganization to break
down their horizontal functional boundaries. Now the focus is on improving
business management and processes across traditional business functions
(for example, customer service, campaign management, risk management,
inventory, and supply chain management). Unfortunately, users must constantly
access multiple applications when dealing with these new processes.
Deploying an ODS is an excellent approach, especially when this challenge
cannot be met by the existing operational applications, the DW, or the datamarts.
The ODS is not just the data store but the population around it. The front ends
and the ODS will provide access to an integrated, consolidated picture of the
organization’s information and may leverage existing IT investments. This allows
organizations to improve or close the loop on business processes.
Definition of an ODS
An ODS is an environment where data from different operational databases is
integrated. The purpose is to provide the end user community with an integrated
view of enterprise data. It enables the user to address operational challenges
that span over more than one business function.
Its principal key differentiators are the update frequency and the direct update
paths from applications, compared to the controlled update path of data
warehouse or datamart.
When to use a separate datastore?
It is worth discussing briefly when to use a separate data store, the ODS —
rather than simply extending the use of existing systems; for example, customer
master files or a data warehouse.
There is no clear-cut rule for this, and in some cases, it is possible to meet new
new real-time requirements by using existing systems.
Many tasks of a system designer or architect involve making trade-offs between
conflicting requirements, as well as anticipating how these requirements are
likely to change over time. Requirements include response time, throughput,
ease of development, cost, service levels, resource utilization, availability, fault
tolerance, serviceability, and so on.
An application database that is extended to provide ODS type function is likely to
suffer from constraints imposed because it was originally designed as an
application specific database. This may limit its performance, or if the ODS is
changed to support ODS function, this may adversely impact the original
application.
Currency of data
An ODS is a current view of the data. It contains current or near-current data. It
may be accurate nearly up-to-the-second, but does not have to meet this
standard. That depends on the objectives of the ODS, the user needs, and the
business requirements for velocity. For example, a risk management ODS in an
investment house may only be updated once to cover off exposures in the
overseas markets. If a business condition changes, for example, parts are taken
from stock, then the operational data related to that condition quickly changes
too. The ODS reflects the current status of a business condition, in this case, the
number of parts in stock.
Updating/loading data
In an ODS, altering (insert, update, delete) an existing record is a normal and
often-used process, because just one status of a business condition is reflected.
In the DW, the records are not updated, but a new snapshot or event may be
created. This difference is shown in Figure 1-2. Multiple versions of data may
exist and this depends on requirements or purpose.
Summarization of data
The ODS contains primarily detailed data. It may contain summarized data for
performance reasons, but this is not its first objective, because it is difficult to
keep the summary data current and accurate.
Therefore the summarized data is normally calculated at request. Because of its
short effective life it can be called “dynamic summary data”. Its value depends on
the moment of calculation. For example, consider the collective account balance
of a company. This is calculated as a summary of all single accounts of the
company. If any of the single accounts changes its value, the summary also
changes.