Build an FDE portfolio project: messy customer data to accepted deployment

01 / CUSTOMER BRIEF

The assistant must not guess which account it serves.

Imagine a B2B support platform with two independently operated tenants. A CRM and an onboarding spreadsheet both export company records. A future support copilot will need the correct customer identifier before retrieving account-specific context. Unfortunately, references are missing, names arrive in different formats, subsidiaries share domains, and a stale reference sometimes points at a different company.

The support lead wants less manual reconciliation. The security owner wants conflicting evidence to stop the automatic path. The operations owner wants to explain a decision and restore a previous release. These are the fictional stakeholders in the downloadable customer brief, not people we interviewed.

The unit of work is a proposed link, not a destructive merge. We never rewrite a customer master, move invoices or grant access to account data. If you cannot separate “I think these records belong together” from “I have changed the system of record,” you have made rollback much harder before writing the first line of code.

Three discovery answers that change the architecture

1. References are tenant-scoped. Both tenants can have a customer called C000. A global lookup is invalid. In a real service the tenant must come from authenticated server-side identity, not an editable spreadsheet cell. Our tenant field is fixture metadata, not access control.
2. A domain is not an entity identifier. Two directory entries intentionally share a domain. Selecting the first match makes directory order part of the business rule.
3. New companies are legitimate. A row with no candidate should remain unmatched. Automatically attaching it to the nearest known company would turn absence of evidence into false certainty.

Before adapting this for a customer, also ask who can correct the master, who owns review decisions, which downstream action a link unlocks, and what happens when nobody clears the queue. Those answers are implementation inputs, not paperwork to write after the demo.

02 / ACCEPTANCE CONTRACT

Write the rejection conditions before writing the matcher.

The exercise uses six explicit release gates. They are teaching assumptions for this dataset, not industry standards. A real customer would negotiate them using the cost of a wrong link, representative data and the team’s review capacity.

Zero wrong links alone is not enough: a policy that refuses every row would pass that test. The coverage gate prevents that trivial solution. Likewise, an impressive overall match rate is not enough: a policy that links everything can still fail every important conflict case.

There is also a gap in this contract worth noticing. The review threshold measures one queue. It says nothing about how long a review takes, or the work required to onboard unmatched accounts and repair quarantined rows. We expose that gap below instead of claiming that passing the threshold proves operational readiness.

03 / IMPLEMENTATION

Four outcomes, with no silent fallback.

The input is JSON: recordId, tenant, externalRef, name, domain and notes. We chose structured exports to keep CSV parsing and model extraction out of the first experiment. Every field has a type and length boundary. Unknown fields are rejected, preventing a truth label from accidentally becoming a matching feature.

Normalization uses Unicode NFKC, trimming, repeated-space collapse and lowercasing. It does not strip arbitrary punctuation, infer legal suffixes, repair URLs or perform fuzzy name matching. A domain field containing https://company.example/path is rejected rather than silently reinterpreted. Expanding normalization is a policy change: test whether two formerly distinct companies collapse to the same key.

The baseline: reference, otherwise first domain

The baseline shares the same validation, normalization and tenant boundary as the conservative policy. Within a tenant it takes an exact reference if present; otherwise it takes the first exact domain match. It has no conflict check and no review state. This is a deliberately simple comparator, not a benchmark against a commercial matching product or a tuned model.

The conservative policy: corroborate or abstain

• A known reference can link automatically only if a name or domain corroborates it, and every supplied name/domain agrees. Any conflict goes to review.
• An unknown supplied reference goes to review. It does not disappear just because the domain looks convenient.
• With no reference, a unique exact normalized name-and-domain pair can link. Neither field alone is enough.
• Some evidence but no unique agreement means review. No candidate means unmatched. Invalid input means quarantine.

The name-and-domain rule is still an assumption. Those fields are not independent proof of legal identity. Two legitimate entities can share both. The code refuses ambiguity when both are present in the master, but cannot discover an entity that the master omits. The directory’s completeness and ownership remain part of the trust boundary.

Each output records a source record ID, outcome, reason code, linked entity or null, and policy version. Free-text notes never influence a decision and never enter the decision ledger. One fixture contains an instruction to ignore policy and reveal invoices; ignoring it proves this deterministic boundary, not general prompt-injection resistance.

Where an AI component would belong

An optional next stage could extract proposed fields from an email into this schema, with source spans for every extracted value. Evaluate extraction separately: omitted conflicts and invented references matter more here than fluent summaries. The extractor must not authorize the link or change tenant scope. No such extractor, LLM call or AI accuracy result is included in v1.0.0.

04 / EXPERIMENT

More automatic links were not better links.

The directory contains 40 synthetic companies across two tenants. Each company contributes six scenarios: clean reference, formatting noise, name plus domain, domain only, conflicting reference, and a new account. Twenty invalid-tenant rows and twenty shared-domain rows bring the total to 280. Truth lives in a separate file and is only passed to the evaluator.

The baseline links 220 rows automatically. Of those, 156 agree with fixture truth and 64 do not: 40 stale-reference conflicts, two name-and-domain rows and two domain-only rows that select the wrong shared-domain company, and twenty explicit shared-domain cases. Its automatic precision is 156 ÷ 220 = 70.9%, not 156 ÷ 280. Its automatic coverage is 220 ÷ 280 = 78.6%. Precision and coverage have different denominators.

The conservative policy links 120 rows, all correctly in this suite. It reviews 100, leaves 40 unmatched and quarantines 20. The row-by-row transition matters: 62 wrong links move to review, two wrong links are corrected using the name-and-domain pair, and 38 previously correct links also move to review. Those 38 are the price of refusing weak evidence: a link can happen to be correct without being justified by the information supplied.

The workload denominator changes the business conversation.

The review queue is 35.7% of input. But 57.1% of all rows are not automatically linked: (100 + 40 + 20) ÷ 280. Those other 60 rows did not vanish. If operations owns both new-account creation and source-data repair, quoting only the review rate understates the work outside automation.

A useful planning equation is minutes = 100r + 40n + 20q, where r, n and q are measured handling minutes per review, new-account assessment and quarantine repair. It is a workload model, not an observed saving. Until those times and the arrival rate are known, do not promise that this policy reduces headcount or fits an existing team.

05 / INSPECT THE EVIDENCE

Look at the row the baseline gets wrong.

This small inspector executes the same matching code against the same synthetic fixtures in your browser. It sends no records to an API. It is not an upload form and does not accept private customer data.

The downloadable project also works without this browser inspector.

For the stale-ID case, the reference says one company while the supplied name and domain say another. Returning a low-confidence link still creates a link. The conservative policy returns entity: null and reason: reference_conflict. A downstream caller must handle the review state explicitly.

06 / REPRODUCE AND BREAK IT

From ZIP to a running, inspectable release.

Download and extract Customer Link Lab v1.0.0. Install Node.js 24 or later, then open a terminal in the extracted project directory. There are no npm dependencies to install and no external accounts to connect.

npm test
npm run evaluate
npm run release
npm start

Open http://127.0.0.1:4173. The local viewer displays the accepted snapshot, release gates and a filterable decision ledger. Stop it with Ctrl+C. The server intentionally listens on your computer only; do not expose it as a service for private customer data.

The checked version passes 28 tests, including input and master ordering, duplicate IDs, tenant separation, conflicting evidence, malformed fields, hostile-note isolation, repeated runs, a blocked release, snapshot tampering, rollback and HTTP behavior. The test runner is Node’s built-in runner; the suite is readable in acceptance.test.mjs.

Now deliberately attempt the bad release:

node cli.mjs release --baseline

It exits unsuccessfully with RELEASE BLOCKED and names the failing gates. It must leave the active snapshot unchanged. That negative test is part of the deliverable: a release gate that has only ever passed has not demonstrated that it protects anything.

What the files let you verify

• Input data contains the master and incoming records. Truth labels are separate.
• The full report contains both policies, every decision, scenario slices and gate outcomes—not only headline metrics.
• The matcher contains the decision rules; the evaluator contains the acceptance criteria.
• The delivery scorecard distinguishes implemented, tested and still-unaccepted work.
• Checksums identify the downloadable release files. Compare your output rather than trusting a screenshot.

Turn the reference into your own portfolio evidence.

Copying the finished project proves very little about your judgment. Before changing the policy, have another person write a private set of cases and adjudicate the intended outcomes. Include domain changes, same-name subsidiaries, incomplete master records and source-system disagreement. Keep that evaluation set out of policy tuning.

Then submit a short decision memo: what changed, which slice improved, which previously valid behavior regressed, what new workload appeared, and whether you would ship. Preserve a failing case and a rejected release. Those artifacts give an interviewer something to question and reproduce beyond “I built an AI app.”

07 / DELIVERY

A green scorecard cannot average away a red safety gate.

The release command reruns the six evaluation gates, records hashes of fixtures and relevant runtime source, writes a content-addressed snapshot and updates a small active pointer. The viewer serves that accepted snapshot rather than recomputing a different result on every request. Releasing identical inputs and code is idempotent. The test suite must also be run separately; the release command does not magically replace CI.

A corrupted snapshot makes the viewer fail closed with a 503 response. npm run rollback switches to a previous accepted snapshot when one exists; with only one distinct release it refuses without modifying state. The automated rollback test creates two accepted snapshots and verifies restoration. This is a single-writer, local demonstration, not a distributed deployment system.

Rolling back code is not rolling back consequences. Restoring yesterday’s matching policy cannot unsend an email or reverse data already disclosed. This project avoids that problem by producing a read-only proposal ledger. If you add CRM writes, design transactions, an audit trail and compensating actions before calling the system reversible.

A real pilot should begin in shadow mode, with no link-driven writes or access grants, followed by a bounded rollout with a control group, named owner and stop conditions. Google’s canarying chapter describes the release-and-evaluation pattern; our local snapshot exercise is not a production canary. The runbook maps wrong links, review spikes and integrity failures to concrete stop actions.

08 / LIMITATIONS

The useful result is a falsifiable contract—not a universal accuracy claim.

The fixtures and the policy were developed together. The classes repeat controlled patterns and deliberately overrepresent conflicts. They are neither independent observations from a customer population nor a held-out generalization test. Therefore “zero wrong links in 120 automatic decisions” is a fixture result, not evidence of a zero real-world error rate. We do not attach a population confidence interval to these authored cases.

Other limits are material: the matcher does linear directory scans; it has no fuzzy or multilingual identity evaluation; the master is assumed authoritative; the release pointer has no concurrent-writer lock; hashes are not signatures; there is no review approval workflow, external CRM, retention policy, load study or independently reviewed security design. The downloadable README calls these out so the boundaries travel with the code.

For an AI extension, evaluate and govern the model-assisted step as a separate component with accountable human roles. NIST’s AI Risk Management Framework is a useful reference for that broader work; this small project is not a claim of NIST compliance.

The contribution here is narrow and reproducible: a deliberately difficult dataset, a transparent failed comparator, an abstaining implementation, observable workload trade-offs and a release that refuses known bad behavior. A reader can change the rules, falsify a result and inspect the consequences. That is the kind of portfolio artifact worth discussing—and citing.

About the author

Kumar Jiv Chumber is a BAIC alum. Connect with Kumar on LinkedIn →