Project Evidence Standards: How to Prove Delivery

Reading boundary: This chapter separates official facts, the FDE working framework, and claims that still require field evidence. The framework is not an industry certification or compliance standard.

1. Completion is a set of testable claims, not one project status

“The project is complete” often compresses six different claims:

• a capability can be demonstrated;
• a bounded pilot has run with real or near-real conditions;
• a specific version has launched in production;
• target users repeatedly use it in the intended workflow;
• the original business measure has changed;
• a receiving team can operate and improve it without the original delivery team.

These claims may occur at different times. A project may have a strong demo, a real production endpoint, and no adoption. It may show an early outcome while still depending on manual work by the FDE team.

This chapter separates four objects:

Object	Definition
Claim	A testable statement the project asks others to accept
Artifact	A file, log, record, dataset, dashboard, or decision produced by the project
Evidence	An artifact tied to a claim, scope, period, version, method, owner, and reviewer
Acceptance gate	A decision rule agreed before results are interpreted

An artifact named “acceptance report” is not automatically evidence. It becomes reviewable only when a reviewer can determine what it supports, where it came from, which version and population it covers, and which limitations remain.

Every evidence record should include an ID, supported claim, artifact type, scope, source system, time window, owner, reviewer, collection or scoring method, evidence status, publication level, protected location, version or hash, and known limitations.

The record must not copy passwords, API keys, access tokens, personal sensitive data, or exploitable vulnerability details into the card. It should point to protected source material.

2. Framework status and official-source boundaries

This chapter is an FDE editorial and project-review framework. It is not an industry certification, a compliance checklist, or a declaration of conformity with NIST or ISO standards.

2.1 NIST

The NIST AI RMF 1.0 is a voluntary framework. Its Core organizes risk-management outcomes through Govern, Map, Measure, and Manage.[S1][S2]

The AI RMF Playbook provides suggested actions. NIST explicitly describes it as non-exhaustive and not a checklist. Organizations are expected to select actions that fit their resources and context.[S3]

This handbook therefore uses:

• Map as a reference for context, intended use, stakeholders, impacts, and problem evidence;
• Measure as a reference for testing, metrics, thresholds, and monitoring;
• Manage as a reference for treatment, incidents, recovery, and stop decisions;
• Govern as a reference for ownership, policy, documentation, oversight, and transfer.

Completing this evidence card does not mean that a project has “passed NIST.” NIST does not certify this framework.

The NIST AI 600-1 Generative AI Profile is also voluntary and cross-sector. It adds generative-AI risk considerations such as pre-deployment testing, continual evaluation, domain-expert involvement, incident and feedback records, and safety-first thresholds.[S4]

2.2 ISO/IEC

ISO/IEC 42001:2023 is an organizational AI management system standard. ISO/IEC 23894:2023 provides guidance on AI risk management.[S5][S6]

Their public materials inform the need for ownership, risk records, monitoring, review, and continual improvement. A single-project FDE evidence card is not an ISO/IEC 42001 management-system audit and cannot prove organizational conformity or certification.

2.3 OpenAI and Anthropic

OpenAI’s official evaluation guidance emphasizes defining the objective, building datasets and metrics, comparing versions, and evaluating continuously. Its agent material extends this to traces, graders, tool calls, workflow decisions, and end outcomes.[S7][S8]

Anthropic’s agent-evaluation guidance emphasizes task-specific evals, explicit success and failure boundaries, combinations of deterministic, model-based, and human graders, real environment outcomes, agent transcripts, the evaluation harness, and long-term production monitoring.[S9][S10]

These are engineering methods, not universal quality certifications. Every project must calibrate them to its users, failure costs, system boundaries, and risk authority.

2.4 OWASP and MCP

The OWASP GenAI/LLM Top 10 provides threat categories such as prompt injection, sensitive-information disclosure, supply-chain risk, insecure output handling, and excessive agency.[S11]

MCP’s official security guidance emphasizes least privilege, progressive authorization, token audience validation, a prohibition on token passthrough, correct OAuth boundaries, and clear human confirmation for sensitive operations.[S12][S13]

These sources help a project define threat and permission evidence. They do not turn a meeting note or a tool choice into a security certificate.

3. Keep three dimensions separate

3.1 The five evidence layers describe which part of delivery is being proven

1. Problem evidence
2. Solution evidence
3. Quality evidence
4. Operational evidence
5. Outcome and transfer evidence

3.2 Editorial evidence levels E0-E4 describe the strength of a public conclusion

Level	Meaning
E0	Opinion or hypothesis only
E1	Traceable official material, research, law, or specification
E2	Project-process artifacts are available
E3	Baseline, outcome, observation window, and measurement method are available
E4	E1-E3 are present and at least one project participant has confirmed the facts

Complete project layers do not automatically create an E4 public case. A vendor-only claim with no source review may remain below E4. Conversely, an E1 NIST principle does not prove that a particular project launched.

3.3 Evidence status describes the current condition of one record

Status	Meaning
Missing	Required evidence does not exist
Planned	Collection method, owner, and date exist, but evidence has not been produced
Observed	Material has been seen, but source, method, scope, or ownership has not been verified
Verified	Source, scope, period, version, method, gate, and appropriate confirmation have been checked
Contested	Data, interpretations, or responsible parties disagree
Not publishable	Evidence can be reviewed by authorized parties but cannot be made public

Not publishable is a publication property, not a quality failure. Evidence can be both Verified and Not publishable.

Verification is time-bound. A model, dataset, workflow, permission, tool, or production-version change can invalidate old evidence.

4. Demo, pilot, launch, adoption, outcome, and transfer are different claims

Claim	Minimum meaning	Minimum evidence	Invalid substitute
Demo	A capability works under controlled inputs, operator, and environment	Solution scope, demo samples, failures, version	A smooth video does not prove field use
Pilot	Key assumptions are tested with bounded real users or data and a defined time window	Problem, solution, and quality largely Verified; bounded operational evidence Observed	An internal test or customer presentation
Launch	A specific version runs in production with real identity, permissions, monitoring, rollback, and ownership	Quality gate passed; production controls Verified	A reachable URL or HTTP 200 response
Adoption	Target users repeatedly complete intended real tasks	User denominator, real task volume, repeat use, coverage, workarounds	Registrations, page views, training attendance
Outcome	The original measure changes against a baseline	Comparable before/after data, window, sample, cost, risk, attribution limits	Positive feedback, model accuracy, usage growth
Transfer	The receiving team accepts ownership and can operate, evaluate, recover, and change the system	Ownership, controlled access, assets, training, drills, accepted open risks	Sending documentation or holding one training session

A precise claim should include version, population, time, and exclusions. For example:

Version v1.3 launched for two business groups on July 1, 2026. The production gate has passed. Adoption remains inside a four-week observation window, and no business-outcome or transfer claim is made yet.

5. Layer 1: Problem evidence

Required fields

• target users, affected parties, and workflow owner;
• trigger, inputs, decisions, actions, and terminal state of the current workflow;
• baseline values with denominator, source, method, and time window;
• current loss, risk, delay, cost, or opportunity;
• current alternatives, including manual work, rules, existing software, and no-AI options;
• intended use and explicit out-of-scope use;
• sponsor, business owner, acceptance authority, and risk owner;
• success, pause, and stop conditions.

Status and acceptance gate

Problem evidence is Verified only when a real user or workflow owner has reviewed the process, the baseline is reproducible, decision ownership is explicit, intended use and exclusions are documented, and material disputes remain visible.

Problem gate: the team can explain in one page who has the problem, in which workflow, what the current measure is, what will change, who can decide, and when the project should stop.

Counterexamples

• “Build an enterprise knowledge agent” with no user or workflow.
• Market-size statistics used as the organization’s baseline.
• Executive interviews with no observation of operating users.
• An average with no denominator, long tail, or failure consequence.
• A demo built first and a matching problem invented later.

6. Layer 2: Solution evidence

Required fields

• solution scope and explicit exclusions;
• alternatives considered and architecture decision records;
• actual architecture, data flow, model, tool, library, protocol, and dependency versions;
• data ownership, permitted use, freshness, deletion, retention, and quality limits;
• identity, authorization, autonomous-action level, and human approval points;
• threat scenarios, assumptions, dependencies, and accepted risks;
• rollback, degradation, compensation, and safe-stop behavior;
• productizable components and customer-specific customization.

For MCP or comparable tool protocols, record least-privilege scopes, token issuer and audience, audience validation, prohibition of token passthrough, separate downstream credentials, sensitive-action confirmation, OAuth and redirect controls, and the boundary between audit logs and secrets.

Status and acceptance gate

Solution evidence is Verified when documentation matches the implementation; a reasonable alternative has been compared; risks and assumptions have owners; agent autonomy is justified by the task; least privilege and audience validation are enforced; token passthrough is prohibited; sensitive actions require testable human or organizational approval; and rollback was designed before launch.

Solution gate: a reviewer can trace every major design choice back to problem evidence and identify how the system stops, degrades, or rolls back when assumptions fail.

Counterexamples

• An architecture diagram containing only an LLM, vector database, and agent box.
• Treating MCP adoption as proof that authorization is secure.
• Passing a client token directly to a downstream service.
• Giving tools broad write access and relying on a prompt to prevent misuse.
• No recorded alternative to an agentic design.

7. Layer 3: Quality evidence

Required fields

• task definition, allowed actions, and successful terminal state;
• task-specific success, failure, and severity boundaries;
• dataset sources, version, population, sample size, and relationship to production;
• deterministic, model-based, and human graders with calibration records;
• trial count, variance, long-tail failures, and uncertainty;
• thresholds defined before interpretation and an authority for exceptions;
• final-state checks, trace checks, tool-behavior checks, and harness version;
• prompt-injection, data-leakage, over-permission, excessive-agency, token-boundary, and human-approval tests;
• known limitations, unsupported cases, regression results, and continuous-eval plan.

Status and acceptance gate

Quality evidence is Verified when success and failure were defined before the main result; the dataset has a defensible relationship to production; graders were calibrated with domain experts; agent evals inspect both the environment outcome and the trace; stochastic tasks use multiple trials; security and permission failures are tested; critical exceptions have an authorized owner; and the tested version matches the release candidate.

Quality gate: every hard gate passes. Missing or contested evidence about critical safety, authorization, or outcome integrity cannot be averaged away. A conditional pass must include a limited scope, expiry date, human control, and exit condition.

Counterexamples

• Three curated success prompts.
• The agent’s own statement that it finished the task.
• Easy-only datasets with no production failures or adversarial cases.
• Thresholds lowered after results without retaining the change history.
• A single average with no severe-error count.
• An LLM grader treated as ground truth without human calibration.
• A changed model or tool version with no regression run.

8. Layer 4: Operational evidence

Required fields

• production release identity and enabled scope;
• rollout strategy, canary or phased release, rollback, and degradation controls;
• SLIs/SLOs for availability, latency, task completion, quality, human review, and cost as appropriate;
• logs, traces, tool-call audit, alerts, escalation, and on-call ownership;
• production eval sampling and drift or regression detection;
• target-user denominator, real task volume, repeat use, coverage, abandonment, and workarounds;
• cost, capacity, quotas, incidents, postmortems, and corrective-action status;
• change records connecting model, data, prompt, permission, tool, and code changes to evidence.

Production and adoption gates

The production gate requires traceable release identity, real authorization controls, audience validation, protected credentials, telemetry, audit, rollback, incident ownership, approved scope, and working human confirmation for sensitive actions.

The adoption gate requires a target-user denominator, repeated completion of intended real tasks over an agreed window, separation of real use from tests and training, and evidence about workarounds, abandonment, manual review, and added operating effort.

Operational evidence is Verified only when production data can be reviewed, rollback or incident procedures have been exercised, production monitoring can catch issues missed by offline evals, test traffic is excluded from adoption, and negative operational signals remain in the record.

No alerts is not reliability evidence when the system lacks observability.

Counterexamples

• Declaring launch because a service is reachable.
• Treating total requests as completed user work.
• Counting delivery-team tests as adoption.
• Reporting uptime without task quality, human review, or cost.
• Privately fixing incidents with no timeline or action tracking.
• Logging full tokens, personal data, or secrets in traces.

9. Layer 5: Outcome and transfer evidence

Required fields

• outcome metric mapped to the original baseline;
• baseline and observed value, denominator, sample, source, and observation window;
• comparison method, concurrent changes, and attribution limits;
• total model, infrastructure, review, correction, training, and migration costs;
• side effects, transferred risks, severe failures, and stakeholder disputes;
• decision to expand, maintain, modify, hold, or stop;
• named owners for operations, evals, security, data, and business outcomes;
• transferred repositories, runbooks, dashboards, datasets, graders, thresholds, vendor relationships, and evidence index;
• controlled re-authorization of credentials, not plaintext secret transfer;
• independent release/configuration, eval-regression, and recovery drills by the receiving team;
• explicit acceptance of open risks, limitations, support boundaries, and technical debt.

Outcome gate

An outcome claim requires a comparable baseline, denominator, period, source, cost, side effects, attribution limits, business-owner confirmation, retained disputes, and a decision rule agreed before the result was interpreted.

Transfer gate

A transfer claim requires actual ownership and access, named responsibilities, controlled credential re-authorization, transfer of operating and evaluation assets, independent drills by the receiving team, and explicit acceptance of open risks and support boundaries.

Outcome and transfer are separate gates. A project may state “an outcome has been observed; transfer is incomplete” or “operational ownership has transferred; the business outcome remains under observation.” It must not claim full delivery when either statement is unsupported.

Counterexamples

• Using model accuracy as the business outcome.
• One positive month with no baseline seasonality.
• Ignoring added review, data-cleaning, and operational cost.
• Attributing forced adoption or a retired legacy process entirely to AI.
• Vendor-calculated ROI with no business-owner review.
• Documentation sent, but no receiving-team rollback or eval run.
• Shared administrator accounts treated as transferred ownership.
• Negative evidence removed from a public case study.

10. Gate decisions

Use four decisions:

Decision	Meaning
Pass	All hard gates for the current claim pass
Conditional pass	Work may continue within explicit limits, controls, deadlines, and exit conditions
Hold	Required evidence is Missing or Contested, or risk is not yet acceptable
Stop	Value, risk, dependency, or operating economics do not justify continuation

Do not collapse the layers into one weighted score. A critical authorization failure cannot be offset by good documentation. No baseline cannot be offset by a high demo score.

Required rules:

• a required Missing item blocks Pass;
• a critical Contested item remains visible and blocks Pass until resolved or explicitly accepted by the right authority;
• Not publishable evidence may support an internal gate only when an authorized reviewer actually inspects it;
• every conditional pass has an expiry date;
• material system changes trigger re-review.

11. Publication and redaction levels

Level	Name	Audience
P0	Public	Anyone; authorized methods and non-sensitive aggregate evidence
P1	Public redacted	Anyone; identifiers and sensitive precision removed while methods, period, denominator relationships, and limitations remain
P2	Controlled review	Named peers, customers, or NDA reviewers
P3	Internal	Project and authorized governance roles
P4	Restricted	Minimum-authorized personnel and protected systems only

Redaction must consider not only names but also rare roles, small populations, exact dates and locations, prompts, screenshots, filenames, internal URLs, domains, IP addresses, tenant IDs, contract terms, vulnerability details, credentials, sessions, personal data, and whether the project had authority to use a sample publicly.

Public evidence should preserve the stage being claimed, the stages not being claimed, the observation window, denominator scale, measurement method, acceptance gate, known failures, human controls, evidence ownership, and the distinction between vendor self-report, customer confirmation, and independent review.

If redaction makes the claim impossible to evaluate, narrow or remove the public claim. Do not replace missing evidence with stronger adjectives.

13. Chapter conclusion

The purpose of a project evidence standard is not to generate more documents. It is to constrain ambiguous claims.

A credible FDE project record should:

1. separate demo, pilot, launch, adoption, outcome, and transfer;
2. preserve evidence for the problem, solution, quality, operations, outcome, and transfer;
3. distinguish existence, observation, verification, dispute, and publication limits;
4. define gates before interpreting results;
5. include permissions, safety, failures, incidents, cost, and manual support in the delivery record;
6. allow Hold and Stop to be responsible project outcomes;
7. never present this handbook as NIST, ISO, OWASP, MCP, OpenAI, or Anthropic certification.

The complete delivery claim is not “we built the system.” It is:

We can show what problem it addresses, why it was designed this way, where its quality boundary lies, how it operates, what changed, and who can continue to own it.

---

Action asset

Open Project Evidence Card v1 to record stage claims, five evidence layers, gate decisions, and publication levels.

Sources and Evidence Boundaries

All sources below are official primary sources. Dynamic pages may continue to change; citations should be read with the access date stated in this chapter. Reports of page changes, broken links, or factual errors are welcome.

NIST

• [S1] Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, published 2023-01-26. Accessed 2026-07-19.
  Boundary: A voluntary risk management framework; it is not a certification and does not automatically satisfy requirements for a specific country, sector, or organization.
• [S2] AI RMF Core, NIST AI Resource Center. Accessed 2026-07-19.
  Boundary: Used to understand the outcome structure of Govern / Map / Measure / Manage; the Core entries are not converted into this handbook’s compliance checklist.
• [S3] NIST AI RMF Playbook, NIST AI Resource Center, updated 2025-08-14 on the page accessed. Accessed 2026-07-19.
  Boundary: Advisory, non-exhaustive, and not a checklist; organizations select actions according to their environment.
• [S4] Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, published 2024-07-26. Accessed 2026-07-19.
  Boundary: A voluntary, cross-sector GenAI risk management reference; it is not an FDE project certification or a mandatory compliance standard.

ISO/IEC

• [S5] ISO/IEC 42001:2023 — Artificial intelligence management system, ISO, 2023 edition. Accessed 2026-07-19.
  Boundary: An organization-level AI management system standard; this handbook’s project evidence card is not a management system certification audit.
• [S6] ISO/IEC 23894:2023 — Guidance on risk management, ISO, 2023 edition. Accessed 2026-07-19.
  Boundary: Risk management guidance; this chapter relies only on ISO’s public page and does not claim access to or reproduce the paid standard text.

OpenAI

• [S7] Evaluation best practices, OpenAI developer documentation, continuously updated. Accessed 2026-07-19.
  Used for: Eval goals, datasets, metrics, human calibration, version comparison, and continuous evaluation.
• [S8] Agents SDK — Agent evals, OpenAI developer documentation, continuously updated. Accessed 2026-07-19.
  Used for: Agent traces, graders, workflow-level evaluation, and final outcome checks.
  Boundary: OpenAI platform engineering documentation, not a cross-vendor quality certification.

Anthropic

• [S9] Demystifying evals for AI agents, Anthropic Engineering, published 2026-01-09. Accessed 2026-07-19.
  Used for: Task-specific evals, success and failure boundaries, trials, mixed graders, transcripts, outcomes, harnesses, and the distinction between development evals and production monitoring.
• [S10] Building effective agents, Anthropic Engineering, published 2024-12-19. Accessed 2026-07-19.
  Used for: Workflow-agent boundaries, complexity control, tool interfaces, and agent engineering patterns.
  Boundary: Vendor engineering practice that still requires validation against a specific system and independent project evidence.

OWASP and MCP

• [S11] OWASP Top 10 for Large Language Model Applications, OWASP GenAI Security Project, 2025 edition page. Accessed 2026-07-19.
  Used for: Failure modes including prompt injection, sensitive information disclosure, output handling, supply-chain risk, and excessive agency.
  Boundary: An open security-risk reference; it does not replace organizational security review, penetration testing, or certification.
• [S12] MCP Security Best Practices, Model Context Protocol specification, version 2025-11-25. Accessed 2026-07-19.
  Used for: Least privilege, audience validation, prohibition of token passthrough, OAuth controls, and confirmation for sensitive actions.
• [S13] MCP Authorization, Model Context Protocol specification, version 2025-11-25. Accessed 2026-07-19.
  Used for: Authorization protocol boundaries, token audience, scopes, and server-side validation.
  Boundary: Covers MCP authorization and related attack surfaces only; it does not establish enterprise-wide security or compliance.

Google SRE and Software Delivery

• [S14] Service Level Objectives, Google, Site Reliability Engineering online edition. Accessed 2026-07-19.
  Used for: SLI/SLO design, target service levels, and user-visible reliability evidence.
• [S15] Release Engineering, Google, Site Reliability Engineering online edition. Accessed 2026-07-19.
  Used for: Repeatable releases, build identity, release processes, and rollback evidence.
• [S16] Postmortem Culture: Learning from Failure, Google, Site Reliability Engineering online edition. Accessed 2026-07-19.
  Used for: Incident impact, timelines, root causes, action items, and blameless postmortems.
  Boundary: SRE practice material that should be adapted to project scale, organization, and risk; it is not a single launch checklist.

Remaining Evidence Gaps

• The five-layer project evidence standard and Project Evidence Card v1 are this handbook’s working framework and currently have no endorsement from an external standards organization.
• This chapter has not yet completed E2/E3 validation with real China enterprise projects. Field completeness, review time, and public redaction rules still require case-based calibration.
• This chapter does not define universal cross-industry numerical thresholds. Quality, risk, SLO, adoption, and outcome gates must be set by each project according to impact, users, and responsibility boundaries.
• Requirements under Chinese law, sector regulation, contracts, privacy, and security must be determined separately by authorized owners; this chapter does not provide legal advice.
• OpenAI, Anthropic, MCP, and OWASP pages may continue to change. Later versions will update citations by review date, and reports of broken links or content changes are welcome.
• This chapter is a public Beta. It does not represent completed legal, privacy, security, contractual, or compliance review for any specific project; those determinations remain the responsibility of authorized owners in that project.