Mosha Pasumansky was architect of MDX at Microsoft in 1997. Lot of m's :-) Microsoft have introduced a language to query or manipulate multidimensional data. That was known as multidimensional expression as that will extract data from multidimensional sources. Earlier it was used just by Microsoft. As data started to grow, more and more data anlaysis tools came on scene. and olap became industry standard.

Many applications such as pentaho, cognos, excel, Microsoft Reporting Service.... uses MDX today.

Difference between MDX and SQL

Ofcourse MDX is multidemsional.

SQL takes table as input and produces columns as output. MDX take multidimensional cubes as input and produces multidemsional cubes as result.

Details

MDX can certainly have multiple axis. Here by axes, we mean dimensions. Generally we can have upt 128 axes in MDX.  In MDX we should always use word axes instead of rows or columns, because X axes of input cube can also be used at Y axes in output cube.

So while talking we will use axes, though in MDX we use rows and columns. MDX has many keywords which are common to SQL like "SELECT", "FROM", "WHERE".....

We want output like this

2006             2007          2008

Loan Accounts     10400          9800          11200

Amount Disb.      255M.           230M        263M.

Here M stand for millions. We want to create cube which tells Loan Accounts Opened by a bank in last three years in Indian branch.  MDX for this will be as following:

SELECT
{ [Date].[2006], [Date].[2007], [Date].[2008] } ON COLUMNS
{ [Measures].[Loan Accounts], [Measuers].[Disbursment Amount] } ON ROWS,
FROM LoanDetails
WHERE ( [Branch].[India])

Here,

• [Date].[2006], [Date].[2007], [Date].[2008],[Measures].[Loan Accounts], [Measuers].[Disbursment Amount] represents data sets.

• { [Date].[2006], [Date].[2007], [Date].[2008] } represents is called X axis

• { [Measures].[Loan Accounts], [Measuers].[Disbursment Amount] } ON ROWS represents Y axis

•  LoanDetails represents input cube for this MDX

• ( [Branch].[India]) represents filter or slicer

In above query we have two axes. But MDX can have more axes too. Generally first three axes are columns, rows and pages. But, here we have to keep one thing in mind, columns will always come first followed by rows, and rows followed by pages.

As we have already discussed axes can change easily in MDX.  Some users may want dataset on X axes other may want it on Y axes.

In above MDX if we replace datasets of X axes and Y axes we get output like this

Loan Accounts  Amount Disb.

2006                 10400                255M

2007                 9800                 230M

2008                 11200               263M

There are few elements in MDX which we use it, but how to classify them is a question. Some technologies call them data types of MDX others classify them as elements of MDX. I am not smart enough to choose which is best word to describe them. But I know what they mean. Just trying to share their definition.

1. Data Types. I do not think we need any definition for it. Scalar can be used either as number or string. Generally it has variant data type. So not much to worry :-) As variant help while retrieving though we need to be careful while inserting in out star schema.
2. Identifiers. Cubes, measuers, dimensions and members are identifiers of MDX.  Cube is basically a mulitdimensional array which stores various data analysis in axes. Dimension is one axes or dimension of a cube, it will basically contain few members. In above example [Date] is a dimension of cube LoanDetails. Member is a member of dimension[Date].[2007] is a member. Measuers is userd on term which we are trying to measure. In above example loan accounts for branh we were trying to measuure.
3. Operators. Operators are elements which are used when we want to combine more then one MDX.
4. Members, Tuples and Set. Members is already explained above. Tuple is one step ahead then that, combination of members. Like [Date].[2007], [Measuers].[Disbursment Amount]. And finally set is final stage it is ordered collection of tuples such as {( [Date].[2007], [Measuers].[Disbursment Amount], [Date].[2006]),9 [Measuers].[Disbursment Amount])} We can also call it cross join.
5. Reserved Keywords. Reserved keyword as name suggest are reserved for use of MDX. Few of reserved keywords that had I had used are DESC, SELF_BEFORE_AFTER, SESSION, DISTINCT, STRTOMEMBER.... Do not worry what this keyword stand for, once you starting using MDX for your OLAP you will figure out.  I was also not knowing about all of them in beginning, but as need arises figured out.
6. Functions. Many of functions are present in MDX to carry out repititive required jobs. You can find list of them over here http://msdn.microsoft.com/en-us/library/ms145970.aspx  People have added to this functions as per their needs. Say for example Mondrian which is used by me for data analysis have added functions like StrToTuple & StrToSet.
7. Comments. You can write comments on your MDX. I do not write them always and regret for it after a while :-) Just add -- in front of your text so that, it will be added as comment.

I am using MDX in Mondrian, it really helps me when to mine database. It's fun, have a try.

Arsitektur Data Warehouse
Secara umum, ada tiga macam arsitektur yang sering ada pada data warehouse, yaitu:

1. Standar
   Pada arsitektur ini, data warehouse langsung mengambil data yang berasal dari berbagai data sumber tanpa adanya tahapan pengumpulan di tempat sementara sebelum masuk ke data warehouse.

   [caption id="attachment_148" align="aligncenter" width="414" caption="Arsitektur Data Warehouse Standar"][/caption]

2. Dengan staging area
   Pada arsitektur ini, data warehouse membersihkan dulu data yang berasal dari berbagai data sumber ke suatu tempat penampungan antara (disebut dengan staging) sebelum masuk ke data warehouse. Proses pembersihan ini akan dijelaskan lebih rinci pada proses ETL (akan dibahas pada posting selanjutnya :P ).

   [caption id="attachment_149" align="aligncenter" width="431" caption="Arsitektur Data Warehouse dengan Staging"][/caption]

3. Dengan staging area dan beberapa data mart
   Pada arsitektur ini, data warehouse akan dibagi lagi menjadi beberapa upabagian (subset) sesuai dengan tujuan maupun penggunanya (misal: subset penjualan perusahaan, subset pemasaran perusahaan, maupun subset inventaris perusahaan). Upabagian dari data warehouse ini biasa disebut sebagai data mart. Data mart itu sendiri tidak selalu dipadang sebagai sesuatu yang diturunkan dari data warehouse (pendekatan top-down), tapi data mart itu bisa dipandang sebagai komponen penyusun data warehouse (pendekatan bottom-up).

   [caption id="attachment_150" align="aligncenter" width="488" caption="Arsitektur Data Warehouse dengan Staging dan Data Mart"][/caption]

Catatan:

Hal, yang patut Anda perhatikan adalah bahwa staging itu bisa memiliki beberapa tingkatan (level), jadi data yang dijumlahkan (di-summarize) pada staging bisa memiliki level penjumlahan yang berbeda. Contohnya: staging data transaksi pada data warehouse suatu dealer motor bisa dijumlahkan per transaksi, per pelanggan, atau bisa jadi harian atau bulanan, tergantung dari kebutuhan user terhadap data warehouse.

Bersambung

-KnightDNA-

Counties Galway and Mayo were like the rest of the country last week, a tad wet, but unlike the developed east of the island, flooding was not a problem; a problematic drainage area is called a lake in the west.

This August has been the wettest and dullest I've ever experienced but at least I saw some sunshine earlier in the month thanks to Kristian Raue CEO of Jedox who kindly invited me to visit the company's offices in Freiburg, Germany.  Freiburg is very green in both senses of the word, surrounded as it is by the Black Forest and its well deserved "eco-city" status.  Its also know as the warmest city in Germany, a reputation it thankfully lived up for this visitor from a rain-soaked Atlantic isle.

If Freburg left a positive impression on my mind, so too did Jedox.  The overall impression is of a company which intends to use a combination of quality, vision and the judicious use of open-source to build the Jedox brand into one associated with best-of-breed products and consultancy.  This vision can be seen in the evolution of Palo, from its "good enough" beginnings to its current near-best-of-breed 2.5 version, and from talking to some of those working on the product, best-of-breed status is not that far off.

Likewise, ETL-Server which is currently a Palo only "loader", is to be further  developed into a true ETL tool, while continuing to offer MOLAP-centric specialisms.

I also got a glimpse of the next version of Worksheet Server. "Wow!", is all I can say.

Existing web based spreadsheet products are fine for simple data analysis or basic data capture purposes but cannot compete with their client-based elder cousins when serious datasmithing is required.  Well, from the demo I saw of Worksheet Server in action, that's about to change.  The look and, more importantly, the feel is similar to that of traditional spreadsheets, its interface with Palo is identical to that of the existing Excel add-in, and here's the big one, its open source!  Game-changing or what?

But ...

That might enable me to move a lot of my spreadsheet applications to the cloud, but what about those applications that are more suited to an MS Access type solution?

Then try out WaveMaker. It’s open source and built on industry standards, Hibernate,Spring and the Javascript Dojo framework but has the ease of GUI database development more usually associated with MS tools. The resulting applications are packaged as a WAR file which can be hosted by any standards based Java server (e.g. Tomcat or Jetty).  The latest version makes developing Ajax-fronted database applications even easier with the addition of layout templates.  Its existing ability to automatically bind interfaces to SOAP web services has been extended to REST web services by means of a new WSDL auto-discover tool.  And Chris Keene CEO of WaveMaker also informs me that ...

We are also releasing a cloud-based IDE in October with Amazon - stay tuned...

We launched in February and will be announcing our first 7 figure deal this month. We run on Mac, Linux and Windows and are currently the #1 developer download on Apple.com (http://www.apple.com/downloads/macosx/development_tools/)

Our goal is to make it easy to build rich internet applications without complex coding – kind of a MS Access for the Web.

Jedox and Wavemaker the new breed of open-source businesses...

OLAP (online analytical processing o procesamiento analítico en línea)

Nos referimos a cubos OLAP cuando hablamos de bases de datos multidimensionales, en las cuales el almacenamiento físico de los datos se realiza en vectores multidimensionales. Los cubos OLAP se pueden considerar como una ampliación de las dos dimensiones de una hoja de cálculo. Por ejemplo, una empresa podría analizar algunos datos financieros por producto, por período de tiempo, por ciudad, por tipo de ingresos y de gastos, y mediante la comparación de los datos reales con un presupuesto. Estos parámetros en función de los cuales se analizan los datos se conocen como dimensiones. Para acceder a los datos sólo es necesario indexarlos a partir de los valores de las dimensiones o ejes.

La principal característica que potencia a OLAP, es que es lo más rápido a la hora de ejecutar sentencias SQL de tipo SELECT, en contraposición con OLTP que es la mejor opción para operaciones de tipo INSERT, UPDATE Y DELETE.

Tipos de sistemas OLAP

Tradicionalmente, los sistemas OLAP se clasifican según las siguientes categorías:

ROLAP
Implementación OLAP que almacena los datos en un motor relacional. Típicamente, los datos son detallados, evitando las agregaciones y las tablas se encuentran normalizadas.

MOLAP
Esta implementación OLAP almacena los datos en una base de datos multidimensional. Para optimizar los tiempos de respuesta, el resumen de la información es usualmente calculado por adelantado.

HOLAP (Hybrid OLAP)
Almacena algunos datos en un motor relacional y otros en una base de datos multidimensional.

 Aunque no todo es tan complicado como parece (¿O si?) , Existe software especializado para crear Cubos OLAP como :

Pentaho: ¿Que es pentaho?

Pentaho es la solución BI Open Source líder del mercado y la mejor alternativa a los productos comerciales.
Las soluciones que Pentaho pretende ofrecer se componen fundamentalmente de una infraestructura de herramientas de análisis e informes integrado con un motor de workflow de procesos de negocio. La plataforma será capaz de ejecutar las reglas de negocio necesarias, expresadas en forma de procesos y actividades y de presentar y entregar la información adecuada en el momento adecuado, mediante analisis OLAP, Cuadros de Mando, etc...

 La plataforma Open Source Pentaho Business Intelligence cubre muy amplias necesidades de Análisis de los Datos y de los Informes empresariales. Las soluciones de Pentaho están escritas en Java y tienen un ambiente de implementación también basado en Java. Eso hace que Pentaho es una solución muy flexible para cubrir una amplia gama de necesidades empresariales – tanto las típicas como las sofisticadas y especificas al negocio

Mondrian: ¿Que es Mondrian?

Mondrian es un servidor OLAP que esta escrito en Java.
Permite interactuar con grandes cantidades de datos almacenados en Bases de Datos relacionales, sin necesidad de utilizar complejas sentencias SQL.  

/* Bueno; espero que con toda esta explicacion @_@ segun... les halla quedado claro que es eso de OLAP, pero y que para los que lleguen por casualidad al blog les sea de ayuda esta pequeña recopilacion de informacion, pero de igual forma les dejo algunos enlaces que estan buenos */

Fuentes:

Definicion de OLAP - Wikipedia
Definicion de Cubo OLAP - Wikipedia
Diseño de Cubos OLAP Mondrian con Pentaho

 Enlaces de Interes:

-Introduccion a Pentaho
-Analisis OLAP con Pentaho
-Como Instalar Pentaho
-Como crear un cubo multidimensional OLAP usando Pentaho
-Funciones OLAP en Microsoft Excel
-Microsoft Dynamics AX con OLAP
-Habla el mayor experto sobre OLAP: Nigel Pendse

Some of the major differences are

SQL Server support for OLAP database systems

SQL Server 2005 Analysis Services (SSAS) for SQL server provides unified and integrated view of all your business data as the foundation for all of traditional reporting, online analytical processing (OLAP) analysis, Key Performance Indicator (KPI) scorecards, and data mining.

Advanced features includes

• Translations, Translations provide a simple and centrally managed mechanism for storing and presenting analytic data to users in their preferred languages.
• MDX Scripts, Multidimensional Expressions (MDX) scripts are the new mechanism for defining calculated members, named sets, and cell calculations.

• Business Intelligence Wizards, A set of easy-to-use wizards can help even the most novice user model some of the more complex business intelligence problems.
• Semi additive Measures, This new measure aggregation type for advanced data modeling includes last-nonempty, last-child, first-child, average-of-children, and even by-account-type.

• Data Mining, Analysis Services provides tools for data mining with which you can identify rules and patterns in your data, so that you can determine why things happen and predict what will happen in the future-giving you powerful insight that will help your company make better business decisions.

New features includes

• Microsoft SQL Server 2005 - Interoperability with the 2007 Microsoft Office System
• Microsoft SQL Server Data Mining Add-Ins for Office 2007
• Microsoft SQL Server 2005 Analysis Services Performance Guide

Related Links :-

Microsoft Analyses Services

Things are looking up on the palo front. I don't know if they have updated the old web client allowing for the creation of subsets in the browser but either way this is good news.

Check it out

Update - I've just started the demo and realised that this isn't complete and is at the moment a demo. Still it will be cool to be able to ETL routines in a browser and them executed on a backend server.
In that respect it's a big like snaplogic. Except snaplogic uses flash for it's GUI and python as the ETL engine.