One of our group members on Master Data Central raised the question about using our automated data standardization, normalization, attribution, rationalization and enrichment processes with transactional data – specifically large volumes of variable length of messages between systems.
 
The challenge – the data in the messages had relatively well defined formats, (about 400 different formats) yet had variable content length for any given format.
 
Seems like a job for a rules processing engine, but the catch? New message formats are being identified monthly – at a rate of 20 per month.
 
Ok – blatant self-promotion warning – using our AI based approach to pattern identification our Harmonize® SaaS platform allows people to normalize and standardize information without writing rules. They find the pattern, they break it into pieces and then Harmonize finds similar information records, messages, or packets, (anything that can be represented in bits and bytes) and applies smart filtering to apply the same standardization, attribute/characteristic extraction, data normalization, and data rationalization (aggregation or unique-ing) to the data sets.
 
In general though, the best practices associated with the data cleansing and enrichment associated with many forms of master data can be well leveraged for big data challenges like high performance analysis and reporting, near real-time data enrichment.
 
Using Artificial Intelligence based componentry like fuzzy logic for match and search, or proximity and frequency algorithms for trending and predictive forecasting are just the jumping off point.
 
If you have successfully re-used best practices from your master data management projects in the transactional, big data or theoretical physics – let us know…
 
Recommended reads:
Blog: Strengthening your E-BOM with MDM
E-Book: Sailing smooth through Data Vortex – Master data management 

• Bio
• Latest Posts

Arthur Raguette

Arthur Raguette is the Executive Vice President at Verdantis. Arthur is very passionate about the application of innovative technologies to solve real-world business problems with a strong emphasis on large enterprise solutions. He has more than a decade of experience of working with Software for Master Data Management and Data Governance for multiple domains and across industries. Arthur’s prior technology passions included high performance B2B middleware and hybridized SaaS applications for HR, Employee and Education related domains.

Latest posts by Arthur Raguette (see all)

• On Demand Webinar:Strategically manage data quality in an ERP rollout - August 27, 2018
• Benefits of Material Master Data Management-Part 5 - March 21, 2018
• Benefits of Material Master Data Management-Part 4 - March 1, 2018