Skip to main content
Guys, I am back. I was out for a while, travelling to east coast of US. So where were we? We talked about data profiling, its need and the approach towards it.


Last few days I have gone through a very good, detailed process of supplier normalization, classification, enrichment using a global compendium. I worked with the people in industry who are doing this business since last 25 years and associated with big names in finance, banking, entertainment and packaging.


So once you do the data profiling, you come to know the richness (or dirtness) of the data. Based on which you can estimate your efforts. But what if customer already has a good structure of the data (not good data though)? Initial work is easy. Talk to customer about the data format, input columns, totalling, what all things customer wants to see. Once requirements are frozen, you can go to next step setting up an environment in your system for the customer.


Test the whole setup using the sample data format from the customer. The input format mapping looks like this one -



So now you have inputs and you are also done with the input mapping analysis of the file you received. Whats next ? Next thing is to preprocess the data. Means whatever are the obvious flaws - knows issues - with teh data correct those. These are normally knows issues communicated by customer. Or may be something that you found and wanted to tell customer about it. Once you are on the same page, get ready to go ahead with first step processing of data transformation.
Will talk about it later. Transformation can be done in two ways - using a tool like informatica (www.informatica.com ), you can map source to target data, setup cleansing rules and get the processed cleansed data before going ahead with normalization. If you are not yet ready with the automation thing, just go ahead on writing your own rules. You need to be really good on writing SQL queries and procedures to do this. Also, this works when you know the rules with priorities and details.

Comments

Popular posts from this blog

Bristlecone Webinar on Supplier Risk Management

Purchasing Magazine, Bristlecone and SAP getting together to bring you very good discussion on supplier risk management. Webinar will be held on June 24, 2 to 3 PM eastern Time, USA. Contributors are - Paul Teague from Purchasing magazine , Jason Buch from spendmatters , Naresh Hingorani from Bristlecone - Supply chain leader company and Padmini Ranganathan from SAP. As we always talk about spend visibility, data issues, strategic sourcing - these distinguised speakers will bring out more strategic views to the table on how all this can be achieved to analyze your business risks better, upfront. You can register for event by accessing this link and register See you there.

Master Data Management – Product or Process ?

I have 2 SAP systems and I want to fix my material master, Services Master. I want all that data to be clean, standardized, classified, enriched and load it back to my SAP in next 6 months. What do you suggest ? Chris - one of my key client was explaining during a “solution understanding” call. My sales manager Tom, enthusiastically started talking about new version of the MDM platform by ERP company, tools, technologies, product landscape, licenses etc. After 30 minutes of sales pitch, I could see confusion on Chris’s face clearly. He said - but I don’t want to add any new product in my infrastructure for all this. Can you just implement MDM for me without I adding any new software ?   Both are using MDM implementation as a keyword, but in a completely different context. Chris wants to implement MDM as a process while Tom was trying to sell MDM as a new software. Whats the difference ? Lot I will say. MDM as a product – when you sell a   software license to a...

Data Management - Elephant & Seven Blind Men

I am sure most of you read a story of Elephant and Seven Blind Men. For starters - this is how the story goes - "A number of blind men came to an elephant. Somebody told them that it was an elephant. The blind men asked, ‘What is the elephant like?’ and they began to touch its body. One of them said: 'It is like a pillar.' This blind man had only touched its leg. Another man said, ‘The elephant is like a husking basket.’ This person had only touched its ears. Similarly, he who touched its trunk or its belly talked of it differently." As Wikipedia mentions - This story has been used to illustrate a range of truths and fallacies. At various times it has provided insight into the relativism, opaqueness or inexpressible nature of truth, the behavior of experts in fields where there is a deficit or inaccessibility of information, the need for communication, and respect for different perspectives. The moral of the story is - While one's subjective experience...