Enterprise Performance Management - Data is First Priority

Microsoft is going high on the BI side. Last month it was zoomix - more of a MDM side and today its datallegro. I guess they are trying to sneak more into the enterprise space by doing this. Lets see what happens more. Follow link to find out what it means to you - http://www.intelligententerprise.com/blog/archives/2008/07/what_the_micros.html http://www.zoomix.com http://www.datallegro.com/

Guys, I am back. I was out for a while, travelling to east coast of US. So where were we? We talked about data profiling, its need and the approach towards it. Last few days I have gone through a very good, detailed process of supplier normalization, classification, enrichment using a global compendium. I worked with the people in industry who are doing this business since last 25 years and associated with big names in finance, banking, entertainment and packaging. So once you do the data profiling, you come to know the richness (or dirtness) of the data. Based on which you can estimate your efforts. But what if customer already has a good structure of the data (not good data though)? Initial work is easy. Talk to customer about the data format, input columns, totalling, what all things customer wants to see. Once requirements are frozen, you can go to next step setting up an environment in your system for the customer. Test the whole setup using the sample data format from the custome...

So you got the large / medium enterprise legacy systems or may be a ERP system in your organization and decided to profile the data you have. The first step is to decide what all data you are going to work with. Normally spend analysis has to be done on your procurement, material data. So material master, vendor master, MRV, PO, part master are the ideal candidate to start. While extracting the data you need to very careful as if you miss some critical fields or required fields for analysis - which you will come to know at very later stage and then everything starts from the scratch. E in the ETL process is a large subject in itself to talk. So we will not get into that right now. Here I will assume that, you are there. You identified the fields correctly and then went ahead with the generating those text files - with proper delimiters. :-). And you are ready for running profiling. Here we have two options. A semi automatic database driven approach and fully automatic tool dependant ap...

You have got an enterprise system for your organization since long time. Do you know how much data quality issues you may have in your data? What does that mean ? 1. You may have different date formats - like mmddyyyy, ddmmyyyy, dd-mon-yyyy and so on. 2.You may have unit of measurement (UOM) inconsistencies issues. - Somebody putting the data in inches, one in cms and one in feet. 3. Have you considered the currency conversion and exchange rate issues. 4. And how about the material code ? One coding it as ABC123, other as ABC-123 and another as ABC 123. All are same, but when you search you dont find the material ABC123 and you order another 1000 quanitities, when same material named as ABC 123 is there in your warehouse. 5. Do you know the vendor ABC, AB-C, AB Corp and AB Ltd are same under one group? All these issues are eating in your money. Directly or indirectly. Just under your nose, because they are the issues, you are going through everyday, but just ignoring it due to invisibi...

As Wikepdia says : Data profiling is the process of examining the data available in an existing data source (e.g. a database or a file ) and collecting statistics and information about that data. The purpose of these statistics may be to: find out whether existing data can easily be used for other purposes Give metrics on data quality including whether the data conforms to company standards Assess the risk involved in integrating data for new applications, including the challenges of joins Track data quality Assess whether metadata accurately describes the actual values in the source database Understanding data challenges early in any data intensive project, so that late project surprises are avoided. Finding data problems late in the project can incur time delays and project cost overruns. Have an enterprise view of all data, for uses such as Master Data Management where key data is needed, or Data governance for improving data quality Now this is all we know. The real quest...

SO yes now you know that you need to have data in correct shape - but how to start all the juggernaut? I will tell you - this is a step by step process. So you need to identify your needs first. How to do that ? Profile your data using simple queries on database server like MS SQL server, Oracle etc. There are sophisticated tools available in the market who can profile data for you. What does this profiling meant to you - it analyze your data and will tell you how much dirty or incosnistent data you have. You may have some null values, date format issues, numeric and character data collision issues, binary data like male, female etc. Now once you know dirtiness of your data atleast you know where you stand. Now the next step is to analyze results and estimate the cleansing efforts. I will tell you that soon. Watch Out.

Your organization churns lot of data everyday.. Purchase orders, inventory, vendors, suppliers, material masters, receipt vouchers and what not. Do you have clear picture of all this data? Do you have visibility to what you are spending, how much, to whom ? If you think twice before answering, you need spend analysis on your data. I am going to suggest a simple assessment package by which you can actually assess your system data and decide whether you need a full scale strategic sourcing initiatives, do you need a supply chain rationalization. Look for this space.