Master Data Management: How FDEs Unify Customer and Product Data

What Is Master Data Management?

Master data management is a technology-enabled business discipline in which business and IT work together to ensure the uniformity, accuracy, stewardship, governance, semantic consistency and accountability of an enterprise's official shared master data. That is Gartner's definition, paraphrased closely.

Gartner describes master data itself as the smallest set of consistent, standardised identifiers and extended attributes that describe an organisation's core entities, used across many business processes. Customers, suppliers, sites, hierarchies and the chart of accounts are typical examples.

The key idea is the contrast with transactional data. An order is a transaction: it happens once and is recorded. The customer who placed it and the product they bought are master data: they persist, and many systems refer to them. MDM keeps those shared references consistent.

Why Master Data Management Matters for AI Projects

AI features depend on entities. A recommendation system needs to know which purchases belong to one customer. A support assistant must retrieve the right account. A demand forecast needs one product history, not three. When the same entity appears as several records, the results degrade quietly:

• Split histories. One customer looks like three people with thin histories.
• Wrong answers with confidence. An assistant retrieves the old address or the closed account.
• Leakage and double counting. Duplicates across training and test sets inflate accuracy.
• Untraceable errors. Nobody can say which system's value is "right".

This is a recurring pattern behind why AI projects fail: the model is fine, but the data it relies on disagrees with itself. FDEs are often the first people to see this, because they work directly with the customer's systems.

Customer Master Data vs Product Master Data

The two most common domains behave differently.

Customer data is mostly a matching problem. Product data is mostly a standardisation problem, with a matching problem on top when you merge catalogues from suppliers or acquisitions.

The MDM Process: Six Steps FDEs Follow

There is no single mandated method, but most successful projects follow this order.

1. Pick one domain and name an owner

Start with one entity, usually customers or products, tied to a business outcome. Name a business owner who can settle disputes about what counts as a customer. Without an owner, every disagreement stalls the project.

2. Map the sources

List every system that creates or changes the entity, what identifiers each uses, and which one is trusted for which fields. Build a crosswalk, a table that links each system's local ID to the shared one. If you are still getting data out of those systems, see our guide to AI data pipelines for messy enterprise data.

3. Clean and standardise

Matching works only on comparable values. Normalise names, phone formats, addresses and units first. This overlaps with data cleaning for AI projects, which covers profiling, missing values and format fixes in detail.

4. Match records

Decide which records describe the same real-world entity. See the next section.

5. Build the golden record

Merge matched records into one, using survivorship rules to choose each field's value. See below.

6. Publish, steward and monitor

Deliver the golden record to the systems and AI features that need it, and put people in charge of exceptions. Matching is never finished, because new records arrive daily.

How Matching Works

Record matching, also called entity resolution, comes in three broad flavours.

• Deterministic rules. Exact or near-exact agreement on a strong identifier, such as a tax ID or customer number. Precise but misses records without the identifier.
• Probabilistic or scored matching. Compare several fields (name, phone, email, address, date of birth), give each a weight, and sum the evidence. Pairs above a high threshold merge automatically, a middle band goes to human review, and low scores stay separate.
• Machine-learned matching. Train a model to decide if two records match. The Ditto paper by Li and colleagues treats entity matching as a sequence-pair classification problem and fine-tunes pre-trained language models. According to the abstract, it beat previous state-of-the-art by up to 29% F1 on benchmark datasets and reached 96.5% F1 on a real task matching two company datasets of 789,000 and 412,000 records.

Strong results on benchmarks do not remove the need for judgement on your own data. Whatever the method, FDEs keep three habits:

1. Measure both errors. A false merge joins two different people. A missed match leaves a duplicate. Which hurts more depends on the use case.
2. Send uncertain pairs to people. Reviewers who know the business catch what scores miss. This is the same human-in-the-loop principle behind our guide to LLM guardrails.
3. Keep merges reversible and logged. You will need to undo some.

Survivorship and the Golden Record

A golden record is the single, trusted version of an entity created by combining matched records. Survivorship rules decide which source value wins for each field. Examples:

• Address: most recently updated value from the system customers edit themselves.
• Legal name: the billing system, because it comes from verified documents.
• Email consent: the most restrictive value across systems.
• Product weight: the supplier file, with ERP as fallback.

Write the rules down and get the business owner to sign them off. Keep lineage, meaning which source supplied each field, so any value can be traced back. This also supports the accountability expected in AI governance for forward deployed engineers.

Four MDM Implementation Styles

How the golden record is stored and shared varies. Vendor material from Profisee describes four common styles, which are widely used in the industry:

Profisee's own advice for organisations new to MDM is to begin with consolidation, and it notes that real deployments are often hybrids. That is a vendor view, but it matches common FDE practice: start with the style that does not touch operational systems, prove value, then move toward more control if the business needs it.

A Worked Scenario: Unifying Customers and Products for a Retailer

This is a hypothetical example to show the method; the retailer and its figures are invented.

A mid-sized retailer wants a personalised shopping assistant. Customer data lives in an e-commerce platform, a loyalty app and store point-of-sale. Products come from an ERP and two supplier feeds.

Scope. The FDE proposes customers first, because the assistant needs a single purchase history. A head of customer experience becomes the owner.

Crosswalk. Each system's customer ID is mapped to a new shared ID. Phone number and email are the strongest shared signals.

Matching. Phone numbers and names are standardised. Pairs sharing a phone and a similar name merge automatically. Pairs with a similar name and address but no shared phone go to a review queue of the loyalty team.

Survivorship. Address comes from the app (customer-edited), name from the loyalty sign-up, marketing consent from the strictest source.

Style. Consolidation: the hub publishes a golden customer table to the assistant and analytics, and the operational systems are left alone for now.

Products. In phase two, the team standardises units and categories across the ERP and supplier feeds, flags probable duplicates, and a merchandiser approves merges.

Result. The assistant sees one history per customer. A weekly report shows pending reviews and merges that were reversed, so the team can tune thresholds.

MDM vs Data Cleaning, Data Warehousing and Ontologies

These ideas overlap, which causes confusion.

• Data cleaning fixes errors inside a dataset. MDM decides what the official entity is across datasets.
• A data warehouse organises data for analysis. MDM feeds it consistent entities; shared customer and product tables become the conformed dimensions described in our guide to data modeling and star schema.
• An ontology, as used in platforms such as Palantir Foundry, models business objects and relationships. It still needs reliable identity underneath, so matching and survivorship questions do not disappear.

Common Mistakes

• Starting with every domain at once. Pick one and prove it.
• No business owner. Technical teams cannot decide what a customer is.
• Merging too eagerly. A false merge is often worse than a duplicate.
• Skipping survivorship rules. Without them, the golden record is arbitrary.
• Ignoring privacy. Merging records can combine data the customer consented to share only in one context.
• Treating MDM as a one-off migration. New records, new systems and changing business rules keep arriving.
• Buying a platform before defining the problem. Tools help, but they do not choose rules for you.
• Forgetting downstream users. Tell the teams and models that depend on the IDs when merges change.

Skills to Build

MDM sits between data engineering and business consulting. Useful skills include SQL, Python for matching and profiling, data modeling, stakeholder facilitation and clear documentation. See the FDE project lifecycle for where this fits in discovery and build, how FDEs build enterprise AI platforms for the wider system, and data engineer to forward deployed engineer if you are making that move. For the full picture of the role, read how FDEs help enterprises deploy AI.

‍