Skip to main content

Posts

Approach to Start the Data Profiling

So you got the large / medium enterprise legacy systems or may be a ERP system in your organization and decided to profile the data you have. The first step is to decide what all data you are going to work with. Normally spend analysis has to be done on your procurement, material data. So material master, vendor master, MRV, PO, part master are the ideal candidate to start. While extracting the data you need to very careful as if you miss some critical fields or required fields for analysis - which you will come to know at very later stage and then everything starts from the scratch. E in the ETL process is a large subject in itself to talk. So we will not get into that right now. Here I will assume that, you are there. You identified the fields correctly and then went ahead with the generating those text files - with proper delimiters. :-). And you are ready for running profiling. Here we have two options. A semi automatic database driven approach and fully automatic tool dependant ap...

Why should I Profile my Data?

You have got an enterprise system for your organization since long time. Do you know how much data quality issues you may have in your data? What does that mean ? 1. You may have different date formats - like mmddyyyy, ddmmyyyy, dd-mon-yyyy and so on. 2.You may have unit of measurement (UOM) inconsistencies issues. - Somebody putting the data in inches, one in cms and one in feet. 3. Have you considered the currency conversion and exchange rate issues. 4. And how about the material code ? One coding it as ABC123, other as ABC-123 and another as ABC 123. All are same, but when you search you dont find the material ABC123 and you order another 1000 quanitities, when same material named as ABC 123 is there in your warehouse. 5. Do you know the vendor ABC, AB-C, AB Corp and AB Ltd are same under one group? All these issues are eating in your money. Directly or indirectly. Just under your nose, because they are the issues, you are going through everyday, but just ignoring it due to invisibi...

What is Data profiling?

As Wikepdia says : Data profiling is the process of examining the data available in an existing data source (e.g. a database or a file ) and collecting statistics and information about that data. The purpose of these statistics may be to: find out whether existing data can easily be used for other purposes Give metrics on data quality including whether the data conforms to company standards Assess the risk involved in integrating data for new applications, including the challenges of joins Track data quality Assess whether metadata accurately describes the actual values in the source database Understanding data challenges early in any data intensive project, so that late project surprises are avoided. Finding data problems late in the project can incur time delays and project cost overruns. Have an enterprise view of all data, for uses such as Master Data Management where key data is needed, or Data governance for improving data quality Now this is all we know. The real quest...

Profile, Cleanse, Classify, Enrich your Data to get spend visibility

SO yes now you know that you need to have data in correct shape - but how to start all the juggernaut? I will tell you - this is a step by step process. So you need to identify your needs first. How to do that ? Profile your data using simple queries on database server like MS SQL server, Oracle etc. There are sophisticated tools available in the market who can profile data for you. What does this profiling meant to you - it analyze your data and will tell you how much dirty or incosnistent data you have. You may have some null values, date format issues, numeric and character data collision issues, binary data like male, female etc. Now once you know dirtiness of your data atleast you know where you stand. Now the next step is to analyze results and estimate the cleansing efforts. I will tell you that soon. Watch Out.

Is your data in right shape?

Your organization churns lot of data everyday.. Purchase orders, inventory, vendors, suppliers, material masters, receipt vouchers and what not. Do you have clear picture of all this data? Do you have visibility to what you are spending, how much, to whom ? If you think twice before answering, you need spend analysis on your data. I am going to suggest a simple assessment package by which you can actually assess your system data and decide whether you need a full scale strategic sourcing initiatives, do you need a supply chain rationalization. Look for this space.

Enterprise problem

Accurate spend data is not available. Too many sources – data difficult to aggregate and classify. Supporting master data like contract, vendor is not available in electronic format. Master data is duplicated, dirty and incomplete. Spending analysis approach is too narrow. Spend data is historic , not forward looking. Analysis process is Adhoc. Data not broken down into cost, volume, drivers with baselines, targets, actual. Confusing technology and service market. Fragmented technology market services. Providers solve different problems and involved in diff phases of life cycle. Building hard ROI case is tricky.

Objectives of Spend Analysis

Identify path to spend reduction. Leverage spend across locations. Develop category specific strategies for spend management. Identify and control areas of maverick spend. To accelerate ROI on strategic sourcing / procurement efforts.