AI model, data and system development procedure

AI model, data and system development procedure

AM-01 — Development scope

The AI engineering owner records whether [Organisation] develops a model, modifies an existing model, integrates a model into a system or acquires an AI service. The development plan identifies intended use, requirements, stakeholders, data needs, evaluation criteria, infrastructure, supplier dependencies and release authority. Responsibilities retained by the organisation and those performed by providers are distinguished.

The procedure supplements the ordinary secure development process with AI-specific data, model and evaluation activities. NIST SP 800-218A is used alongside SP 800-218 rather than treated as a replacement. The plan identifies the practices applicable to the actual role and records activities that cannot be performed directly because of supplier or technical boundaries.

AM-02 — Data acquisition and provenance

The data owner records the origin, acquisition conditions, permitted uses, transformations and known limitations of training, testing, fine-tuning and alignment information. Provenance identifies the source and processing history where known and explicitly identifies unknown portions. The owner checks information rights, privacy purposes, quality and security before approving the intended use.

Dataset versions identify included sources, selection criteria, preparation steps and relevant labels. The record identifies who can alter data or labels and how changes are reviewed. Potential poisoning, tampering, inappropriate sensitive information and mismatch with the intended operating population are assessed. Data obtained from a provider is not presumed to have complete provenance because it is delivered in a convenient format.

AM-03 — Prepare and protect the pipeline

The engineering owner identifies training, evaluation, model storage and deployment environments and their access boundaries. The environment policy applies to compute, orchestration, notebooks, experiment tracking, model registries and data stores. The owner records permissions to change datasets, training code, configuration, model artefacts and release records.

Pipeline records identify code, environment, data and model versions, configuration parameters and resulting artefacts. Monitoring addresses security-relevant activity in environments hosting models and related resources. Unexpected model modification, unusual access or changed execution behaviour is investigated through the recorded route. Secrets and sensitive information are handled according to the environment and data policies.

AM-04 — Develop and modify models

The development record states the objective of training or modification, selected approach, starting model, dataset versions, configuration and expected evaluation. Material choices affecting behaviour or risk are recorded as engineering decisions. Experiments are distinguishable from released candidates, and failed or inconclusive experiments retain the information needed to understand the result where required by the evidence plan.

The owner considers whether training and test information are sufficiently separated for the intended evaluation and records possible contamination or leakage. Adversarial or otherwise challenging information is included where selected to assess or improve relevant behaviour, with handling controls appropriate to the content and environment. The record distinguishes a method intended to reduce a weakness from evidence that it succeeded.

AM-05 — Integrate the AI system

The system architecture records the model, prompts or other instructions, retrieval sources, preprocessing, output handling, connected tools and human interactions that jointly produce the capability. Requirements apply to the integrated system. Model output is handled according to its trust and authority context before it is used as code, a command, a record or an external communication.

Integration identifies failure behaviour, resource constraints, unavailable-provider handling and the conditions for fallback or intervention. Changes to retrieval information, prompts, guard mechanisms or tool permissions are versioned where they affect the system’s decision basis. A model-only benchmark is not used as a substitute for evaluating the configured system in its intended workflow.

AM-06 — Supplier models and services

The supplier owner obtains available model and service information relevant to the intended use, including capabilities, limitations, evaluation, data handling, update practices and security contacts. The local record identifies which artefacts and versions can be inspected, retained or reproduced. When a provider changes a remotely managed model without exposing an exact version, the owner records that visibility limit and the monitoring or reassessment arrangement.

Local evaluation addresses the requirements for the integrated use. Provider evidence is credited only for the claim and conditions it supports. Gaps in training-data visibility, testing or provenance remain in the risk record. The decision authority determines whether the available evidence supports the proposed use within the stated conditions.

AM-07 — Release and preservation

The release candidate identifies model artefacts or service identifiers, dataset references, code, configuration, prompts, retrieval state, tool permissions and evaluation results relevant to its behaviour. Integrity and provenance evidence are retained where available. The release procedure records authorisation, limitations, operating responsibilities and change triggers.

The custodian preserves the evidence needed to explain how the candidate was produced and assessed, subject to data handling and retention requirements. Reproducing a training run may not reproduce identical behaviour; the record states the reproducibility level demonstrated and the sources of variability. Replacement or retirement considers model artefacts, derived information, access and continuing supplier obligations.