Why Manufacturing AI Projects Stall After a Good Demo

Picture a 60-person CNC shop that runs a Claude pilot on its quality records. The first 2 weeks look promising. By month 6, it cannot answer "have we ever waived this dimension for this customer before," it mixes up last revision's PPAP with the current one, and the quality director quietly stops opening it. The model is not the problem. The records the shop runs on (BOMs, routings, first article reports, customer quality manuals, AS9100 evidence, outside processor certs, engineering change history) are scattered across places the AI cannot read with the history intact. This post covers why the pilot stalls, which records a shop carries that the AI needs, and the work that gets those records into shape so the AI answers the way your best engineer would.

The pattern in manufacturing

Manufacturers have spent the last 18 months running AI pilots. The pilots usually start with a narrow, plausible use: summarize the morning's quality reports, draft customer responses, pull the data out of inbound RFQs. The demos work. The pilots launch. The first month is encouraging.

Then comes the second month, when the AI is asked questions that span history. Compare this part's CMM data with the same part run 6 months ago. Pull the customer's PPM trend across the last 4 quarters. Find every job in the last 2 years that went through the same outside processor with a heat-treat spec in this range. These are the questions your experienced engineer answers from what they remember and the paper file. The AI cannot, because the records underneath it are a mess, and it is pulling one document at a time from a Google Drive nobody has organized since 2022.

The pilot stalls. The team concludes the AI is not ready. The model is the same one that demoed well. The records are the problem.

The records a manufacturing shop actually carries

Below is what a typical precision machining, fabrication, or assembly shop keeps on file. Each row is a kind of record an experienced engineer works across every day. Each row is one an AI cannot work across while it only sees the single document in front of it.

Record	How much, and in what shape	What an AI does with only the file in front of it
Bills of materials	Hundreds to thousands of multi-level BOMs, versioned, with effectivity dates and ECN history.	Pulls components from one BOM at a time. Cannot see which components are shared across products, cannot trace a component's history.
Routings and standards	Per part family, per work center. Setup times, run times, tooling lists.	Reads one routing. Cannot spot standard times that are off, cannot suggest changes from what jobs actually took.
Quality records	FAI reports, PPAP packages, CMM data, inspection records, MRR / NCR history per part and per customer.	Summarizes one record. Cannot pull defect trends across customers or part families, cannot answer "have we seen this NCR before."
Drawings and revisions	CAD files, customer-supplied prints, ECN records, GD&T notes, revision histories.	Reads the print handed to it. Cannot notice that a customer is on revision F but the floor is running revision E.
Customer portals and requirements	Customer-specific quality manuals, supplier portals, special process specs, PPAP submission requirements.	Reads the manual it is fed. Cannot spot conflicts between 2 customers' requirements or find a customer's past waivers.
AS9100 / ISO evidence	Audit findings, corrective actions, training records, calibration logs, supplier evaluations.	Reads one finding. Cannot prepare an audit response that traces an issue across 3 years of corrective actions.
ITAR / DFARS records	Citizenship attestations, country-of-origin certs, specialty-metals compliance, controlled-tech logs.	Cannot confirm compliance for a new job by checking the evidence already on file.
Outside processor cert chains	Heat treat, plating, anodizing, NDT, passivation. Lot-level certs from outside processors.	Cannot put together a complete cert chain for a job that went through 3 outside processors without someone stitching it by hand.
ERP / open jobs	Open quotes, WIP, on-hand inventory, customer orders, AR / AP. Often in Visual Job Shop, JobBOSS, Global Shop, ProShop, or E2.	Reads the snapshot handed to it. Cannot answer "what's our WIP by customer" without a live connection to the ERP.
Tooling and fixtures	Fixture inventory by part family, tool life records, calibration logs.	Has no way to know which fixtures exist or what condition they are in unless that list is written down where it can look.
Customer scoreboards	PPM, OTD, supplier scorecards customers issue back to the shop.	Sees no trend. Reads each new scorecard on its own.

The pattern: every row is the kind of question a production planner, quality director, or program manager answers by walking to a filing cabinet, opening ProShop, calling the outside processor, and reading a 6-year-old AS9100 audit response. The AI was supposed to make those walks unnecessary. With the records scattered, it makes them worse, because it confidently fills in the blanks your experienced engineer would know to leave open.

The demo

Prompt→ Model→ Answer

The production system

Prompt→ Model→ Answer

State layer

• Decision records, versioned and reviewed
• Tool access to the systems of record
• Retries and handoffs across a shift
• What changed since the last run

What a shop needs in place

The shape is the same in any industry. The contents here are a shop's.

• BOMs and routings with their full revision history. Each part family written up once, in one place, as the version everyone works from. ECN history visible. Routings checked against actual run times by work center.
• Quality records filed by part, customer, and defect type. FAIs, PPAPs, CMM data, MRR / NCR history. Searchable across time and across customers.
• A drawing register that tracks revisions. The customer's current revision, your current revision, and every earlier revision on hand with the date and ECN that drove it.
• A file for each customer. Their quality manual, their PPAP requirements, their special processes, their portal links. Dated, so it is clear which version was in force when.
• Compliance evidence filed by clause and date. AS9100, ISO 13485, ITAR, and DFARS evidence. This is what lets the AI put together an audit-ready answer without someone rebuilding it from the filing cabinet.
• A file for each outside processor, with its certs. Each heat treater, plater, and NDT house: their certs, their lot-level traceability, their record per process.
• A live connection to the ERP. Open jobs, WIP, and inventory read at the moment someone asks. The files hold how the shop does things; the ERP holds where things stand today.

None of these are exotic. The shop almost has each one already, scattered across Visual or JobBOSS, a shared drive nobody curates, a quality manager's local folder, a customer portal that takes 90 seconds to log into, and the compliance lead's filing cabinet. The work is to pull those records together, keep every revision, and have a named person sign off on each change, so the AI reads what the shop actually does.

How the work runs

KoldOps does this work as a fixed-scope contract. It starts with one area where the questions pile up, usually quality records, customer requirements, or routing standards, and one part family or one customer to prove it on. The sequence:

1. A close look at what you run today (2 weeks). On site. Map where the records live and how they move. Check how ready they are for an AI to read. Pick the area where slow answers cost the most hours. Hand back a written report.
2. Getting that area's records in order (4 to 8 weeks). Pull the scattered records into one place, keep every revision, and put a named reviewer on each change. Set up the search the AI uses to read them. Connect the existing ERP for live job data.
3. An AI trial on one part family or one customer (2 to 4 weeks). Point Claude, or whichever leading model you choose, at the records. Ask it real production questions. Compare its answers with what your most experienced engineer would say.
4. The next area. Your team runs the pattern on its own where it can. KoldOps stays on as advisor.

Planned time from signed contract to an AI your people use on the first area: 8 to 14 weeks. What the shop keeps at the end is its own records, in order, with every revision kept, readable by any AI model it picks now or later.

Frequently asked

We already have Visual Job Shop / JobBOSS / ProShop / Global Shop. Why do we need anything else?

Those are systems of record. They are good at what they are for (job tracking, scheduling, costing, MRP). They are not, by themselves, laid out for an AI to read history across. The work above sits alongside the ERP, pulls job data from it, and lays out the records around it so the AI can answer across them. Your ERP stays. This adds to it. If the ERP itself is what holds the shop back, that is a different conversation, and what we build covers it.

We are AS9100 / ISO 13485. Will this help us pass audits?

It helps with the part that eats the time: finding evidence. Auditors ask "show me the evidence for clause 8.5.1," and the system returns the filed evidence with author, date, and review trail for each document. The clauses where it helps most are documented information (7.5), monitoring and measurement (9.1), and corrective action (10.2). It does not replace the QMS itself; it makes the evidence your QMS already holds faster to find.

Our customer-specific quality manuals are PDFs that change every 6 months. Can this keep up?

Yes. Each PDF is turned into plain text when it arrives, both copies are kept, and every revision is saved, so a customer's revision 3 and revision 4 manuals can be compared line by line. When someone asks "what does this customer require for first-article inspection," the AI pulls the version in force, with the date and reviewer attached.

We are ITAR. Can this run on-premise?

Yes. The records are plain text files with full revision history, the search software is open source, and the AI model can run on a Mac Mini cluster or a dedicated GPU box in the plant. RAG for manufacturing covers where to run it and what each option trades. Nothing in this setup requires the cloud.

Can we start with one customer or one part family?

Yes, and we recommend it. The approach proves itself fastest on a narrow area: one customer, one high-volume part family, or one outside processor. Once the AI gives useful answers there, the next area usually costs less than the first, because the pattern is already set.

What's next

If your shop has run an AI pilot that demoed well and stalled, write to Kol with the question it could not answer. That one question usually shows whether the gap is in the records or in the model. Kol reads every message.

If the gap is in the records, the next step is a close look at what you run today. It starts on site and ends with a written map of where your records live and how they move, plus a fixed scope and price for the first piece.

For why the same thing happens outside manufacturing, see why AI demos work and then stall. For the technical detail, see the 5-question audit (15 minutes) and the definition piece behind it.