In brief: a Forward Deployed Engineer is measured by a production outcome the customer owns, not by a demo or by hours on site. Four kinds of evidence answer that test: the system works on real cases, it fails in a way operators can see and stop, the intended people use it, and the target workflow improved against a baseline. Agree those measures during discovery, before the build. Most generative AI efforts never get that far. In July 2025, Project NANDA at MIT reported that 95% of organizations in its study were getting zero return.
What counts as a successful Forward Deployed Engineer deployment?
A Forward Deployed Engineer deployment succeeded when a named customer owner can run the system in production and the agreed outcome has moved against a baseline. The Council’s definition of a Forward Deployed Engineer says the role is measured by production outcomes, not demos or hours. That rules out three common stand-ins. A demo shows that a path is possible. A pilot shows that a narrow slice can run. Time on site shows that someone was present. None of those, on its own, shows that the customer’s work got better and that the customer can keep it that way.
The outcome has to be specific enough to check. “Better decisions” is not a measure. “The time to close a routine claim fell from the baseline, on the cases we agreed, by the date we agreed” is a measure. Domain 2 of the draft framework, workflow design and value definition, is where that target and its conditions for success are set. Domain 6, verification, evaluation and outcome measurement, is where the evidence is gathered.
A successful demo is not a successful deployment. Success is a production outcome the customer can run and measure.
Which four questions should the evidence answer?
For a Forward Deployed Engineer deployment, collect four kinds of evidence and keep them separate. A system can pass one and fail the others. NIST’s AI Risk Management Framework says AI systems should be tested before deployment and regularly while in operation, and that performance should be shown for conditions similar to the deployment setting, not only on a lab set. The same test applies when the system is not AI. A batch job, an integration, or a rules change still needs proof it works where it will be used.
| Evidence | Question it answers | A pass looks like | Not enough on its own |
|---|---|---|---|
| Works | Does it do the job on real cases? | Agreed cases pass, and known limits are written down | A demo on friendly data |
| Fails safely | When it is wrong or down, can operators see it and stop it? | Monitoring, a rollback or disable path, and a named owner | “It worked in the pilot” |
| Used | Do the intended people use it in the real workflow? | Use after launch, in the job it was meant to change | A training session that people attended |
| Improves the work | Did the target outcome move versus the baseline? | Time, errors, cost, or quality compared with the baseline on the agreed date | Activity counts, such as prompts sent or logins |
Read the rows together. A model with a high score on a test set can still fail the “used” row if people route around it. A busy dashboard can fail the “improves the work” row if the original cycle time did not change. Activity is not an outcome. NIST states the same limit in Measure 2.3 of the AI RMF Playbook: performance estimates from test data do not, by themselves, show how a system behaves in the conditions where it is deployed.
Why do so many AI efforts never show a result?
They stop before anyone measures an outcome in production. In preliminary findings published in July 2025, Project NANDA at MIT reported that 95% of organizations in its study were getting zero return on generative AI. The authors set that against $30–40 billion in enterprise investment into generative AI, a figure stated in the report without a separate source table. Just 5% of integrated AI pilots were extracting millions in value. The vast majority had no measurable impact on profit and loss.
A second cut looks at custom or vendor-sold tools rather than general chat tools. Of the organizations studied, 60% evaluated those tools, 20% reached a pilot, and 5% reached production. The authors define a successful task-specific implementation as one that users or executives called a marked and sustained gain in productivity or in profit and loss. The appendix also defines success as deployment past a pilot with measurable indicators, with return checked about six months after the pilot.
General-purpose chat tools look further along. More than 80% of organizations had explored or piloted them, and nearly 40% reported deployment. The authors say those tools mostly raise individual productivity. They are not, by themselves, evidence that a workflow outcome moved. A tool people like can pass the “used” test and still fail the outcome test.
The research ran from January to June 2025. It used 52 structured interviews, a survey of 153 senior leaders, and a review of more than 300 public AI initiatives. The authors say the percentages are directionally accurate from interviews, not from official company reports, that sample sizes vary by category, and that six months can be too short for a complex system. Read 95% as a finding about this sample and this definition of return, not as a law of every AI project.
What the authors say about method matters for a Forward Deployed Engineer. Buyers who got further evaluated tools on business outcomes rather than on software benchmarks. That is the same gap visible in hiring. The Council’s reading of recent posting data found no direct signal for evaluation in Forward Deployed Engineer job postings, even though the work needs it.
When should a Forward Deployed Engineer agree the measures?
Agree the measures during discovery, before a Forward Deployed Engineer commits to a build. The delivery lifecycle starts by turning the request into a problem statement, a baseline, and a testable outcome. If the team waits until launch, it will be tempted to pick a number that flatters the demo.
Five checks keep the measure honest:
- What will change, and what is the baseline today? Record the current figure, who measured it, and the date.
- Which cases must pass, and which failures are unacceptable? Include the messy cases, not only the happy path.
- Who is supposed to use it, and how will we know they did? A training session is not use.
- Who owns it after the FDE leaves, and what will that person watch? Name the owner and the signal.
- On what date will we compare the outcome with the baseline, and what decision follows? Continue, narrow, or stop.
If there is no baseline because the work was never measured, say so and start one. A first measurement is still a baseline. Do not invent a before-figure after the fact. Do not put a count of activity, such as messages drafted, in the same sentence as an outcome, such as claims closed, without saying they are different things.
What should be true at handover?
Handover is part of the measure for a Forward Deployed Engineer deployment, not a separate courtesy. The deployment is not done when the FDE can operate it. It is done when the named customer owner can. Leave the evaluation cases, the baseline, the known limits, and the signal the owner will watch. The lifecycle guide uses the same test: the owner demonstrates the operational tasks and accepts the system.
Plan a date after launch to compare the outcome with the baseline. NIST’s framework asks for measurement before deployment and regularly while the system is in operation, because behavior can change once real users, real data, and real load arrive. A single launch-day screenshot does not answer the outcome question.
How does this sit with the draft framework?
Domain 6 of the FDE Council Professional Capability Framework asks a Forward Deployed Engineer to show whether the solution works, how it fails, whether users can use it, and whether it improves the target workflow, with evidence matched to the risk. Domain 2 sets the target. Domain 7 covers whether the system can keep running. Domain 9 covers whether people adopt it. The four rows in the table are a plain-language reading of that draft, not a new standard.
The framework is a public draft, open for comment. It has not been validated through a formal job task analysis and is not a hiring test, certification, or credential. There is no FDE certification yet. Use the four questions as a working checklist, and set the threshold to the risk of the system in front of you.
Frequently asked questions
What is a successful FDE deployment?
A successful Forward Deployed Engineer deployment is a system in production that a named customer owner can run, with an agreed outcome that improved against a baseline. A demo or a pilot is not that result.
Is a pilot a successful deployment?
No. A pilot shows that a narrow slice can run. Success requires production use, a customer owner, and a measured change in the target workflow.
What baseline should an FDE record?
Record the current value of the outcome you intend to change, who measured it, and the date, before the build. If that work was never measured, start the series now and say there was no earlier figure.
Does every deployment need a financial return?
No. Match the measure to the problem: time, errors, cost, quality, or another figure agreed up front. A financial return is one kind of outcome. State which figure you are using so a later reader does not mix them.
Who decides whether the deployment succeeded?
The people who own the outcome, using the measures agreed at the start. The Forward Deployed Engineer makes that evidence visible, including when the result is disappointing.
Sources
- Project NANDA at MIT, The GenAI Divide: State of AI in Business 2025, preliminary findings by Aditya Challapally, Chris Pease, Ramesh Raskar, and Pradyumna Chari, July 2025. Research period January to June 2025. Method: 52 structured interviews, a survey of 153 senior leaders, and a review of more than 300 public AI initiatives. The $30–40 billion investment figure is stated by the authors without a separate breakdown. Accessed November 6, 2026.
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST.AI.100-1, January 2023. Measure function. Sample size not applicable. Accessed November 6, 2026.
- NIST, AI Risk Management Framework Playbook, Measure 2.3. Publication date not stated on the page. Sample size not applicable. Accessed November 6, 2026.
- FDE Council, What is a Forward Deployed Engineer?, the Professional Capability Framework draft v0.2, and the delivery lifecycle.
How to cite this article
FDE Council. (2026, November 6). How do you measure whether an FDE deployment succeeded? https://www.fdecouncil.com/insights/measure-fde-deployment-success/. Please also cite the original source of any figure you reuse.
Basis: FDE Council’s role definition and draft framework v0.2, plus the NIST and Project NANDA sources listed above, all accessed November 6, 2026. The four-row table is the Council’s reading of that draft, not a validated standard. The 95% figure is the study’s finding for its sample and its definition of return. The Council’s text and tables are licensed CC BY 4.0; third-party figures remain under their sources’ terms. Not employment or financial advice.
More FDE Council Insights · Read the open competency framework
Related: How to map a learning path to the framework · What the FDE delivery lifecycle looks like · What FDE job postings ask for · How employers can assess FDE skills.