Oracle BI & SOA - Hype or Here Now?

BI is positioned very much within the Fusion Middleware stack of products, alongside areas such as identity management, as one of the key pillars of the platform, and many of the BI Suite Enterprise Edition talks made reference to how the suite played well with SOA-enabled applications. So what does this mean then, in practical terms, for Discoverer, Warehouse and Oracle OLAP developers? Is it just another architecture, or does this represent a step-change in the way we put BI applications together?

As a basic starter, a Service Orientated Architecture is an application architecture that makes use of loosely-coupled, self-describing web-based components that can be accessed by a calling application without any knowledge needed of the way in which they are implemented. Like previous component architectures used when building VB, C++ or Java applications, you can call published components and work with the results, except in the SOA world there's no need for the calling application and the components being used to be built using the same technology; you just register the components using something called a broker, your calling application then checks in with the broker, establishes what services are available and what parameters they require, then makes use of them as desired.

What we're doing then is building an application architecture at the macro, rather than the micro, level. It's evolutionary, rather than a complete new way of doing things - components and object-orientated programming have been around for a while now, but the difference here is that by integrating at the published, standards-based interface level, you can string applications together that draw on functionality from all sorts of systems, even where you have no idea as to how they were implemented.

Apart from the loosely-coupled integration factor, another key feature is that these services can be "orchestrated" - the analogy here is with Oracle Workflow, but Oracle Workflow generally works with PL/SQL components only whereas SOA orchestration, using BPEL (Business Process Orchestration Language), is implementation-independent and a standard across the industry. As well as BPEL, another acronym is ESB, or "Enterprise Service Bus" - a communications layer that takes messages from one SOA component to another, queues them and ensures their delivery.

So, where's the relevance for business intelligence and data warehousing? Here are some recent articles on the subject:

• What Do SOA and ESB Mean in Business Intelligence - Colin White
• What Does Software as a Service Mean for Business Intelligence - Colin White
• What Does Web 2.0 Mean to the Enterprise - Colin White
• Integrating Business Intelligence into the Collaborative Workground Environment - Colin White
• Business Intelligence and SOA - Robin Mulkers
• Why SOA is Important to Your BI Solutions - Colin Hulford
• OTN Service-Orientated Architecture Technology Center

Reading through these, it seems that the major areas of impact for business intelligence are:

1. Exposing ETL functionality as a service, having these routines transform data coming into a warehouse in real-time and in such a way that they can be called by any sort of application
2. Surfacing analytics, reports, dashboards and so on generated by SOA-compliant BI tools within line-of-business applications, and
3. Using the scalability of these loosely-coupled applications to create bigger and better BI applications that are quicker to respond to new data sources, new business opportunities and new ways of analyzing data.
4. BI has to become pervasive, user-friendly and as easy to use as Google Search.

So, getting back to the real world, what does this mean for us? Well, my initial thoughts are along these lines:

• Our ETL tools need to be able to consume, and produce, web services, and work in real-time, so that they can provide the data formatting and transformation facilities required in an SOA environment. This is a step-change beyond the "flat files and database links" that we're currently used to with OWB, and we're going to have to get used to writing ETL routines that are fast, can be run on-demand as opposed to in a batch, and are more about interfaces and standards than automatically assuming we're working with Oracle-only data.
• The analytics generated by our BI tools need to be made pervasive - it's no longer enough to provide a standalone client-server or web-based query tool; we need this tool to make it's insights available, again through web services, to all sorts of applications, so that for example a real-time prediction, or a visualisation showing a customer opportunity, is as likely to be shown in a call centre application as in a dedicated BI dashboard.
• We have to build our data architectures in a way that makes them quick to respond to changes in the data landscape; no more three months to load a new data source via OWB into a data warehouse, we need the data in now, integrated and presented in such a way as to immediately be relevant
• We need to get away from crosstabs, dedicated BI tools that need a week's training to master, and present just the insights people need to do their jobs directly in the applications they work with.

In terms of real-world specifics, my predictions in terms of Oracle BI embracing SOA include:

1. Much more emphasis on BI's place in Fusion Middleware - more emphasis on identity management, more emphasis on interoperability, such that all major new features in Oracle BI (especially Discoverer, OWB) are centred around SOA-interoperability and enabling their use in this new architecture.
2. A very strong emphasis on the technologies now available through the Sunopsis purchase. Sunopsis Data Conductor ticks all the right boxes in terms of SOA-enablement, process orchestration, being platform agnostic and so on. OWB will still be important, but in the world of SOA, Sunopsis will be the key ETL technology.
3. BPEL will be the key workflow technology, replacing Workflow as the way you string together ETL tasks and business processes.
4. BI Suite Enterprise Edition will be the key BI technology, with analytics then being surfaced through traditional dashboards but more increasingly, directly through the Fusion line of business applications
5. The new Real-Time Decisions product will be the key to delivering real-time analyics, and automated decisions, through this architecture
6. OWB will take on more SOA-type features over time; already you can consume a web service through a PL/SQL routine, it's only a small step to publish OWB mappings or processes as web services through a PL/SQL wrapper
7. Web services will increasingly be used to expose Discoverer functionality to other applications
8. In the Oracle BI world, it'll increasingly be about working in heterogeneous environments - Oracle will still hopefully be the centre of the BI world, and will individually have the best features, but it'll play well with other applications and BI solutions developed using Oracle tools will be designed as data source and visualisation technology agnostic

So, it looks like interesting times. Going back to my original question - hype or here now? - I'd say I'm coming down on the "here now" camp. Certainly if you're a partner, or a customer, this is the conversation Oracle and it's competitors are having with their customers, so by definition it's here now.

In terms of Oracle's products, SOA is definitely here in terms of the new BI Suite Enterprise Edition, Fusion Middleware and Sunopsis Data Conductor; it's not such an easy fit when you come to the traditional tools - but I guess that's not a bad thing; if you're after a fairly straightforward, tactical solution, and you're primarily working with Oracle technology, then Standard Edition and OWB are the best fit.

If, however, you've seen the possibilities when it comes to Oracle's middleware platform and the new SOA paradigm, well it just so turns out that Oracle may have the products to make it all work. Interesting times indeed.