Skip to content

AI Impact Assessment

Scored assessment, DMN risk tiers, two-tier human sign-off.

A governed intake path for AI systems inside a federal agency: scored, risk-classified, and gated on a named signature before production.

At a glance
Eight assessment dimensions · 87 scored questions · DMN risk classification · Two-tier human sign-off

The problem

Teams across the agency were starting to propose systems that made or informed decisions about people. What was missing was a way for an institution accountable to the public to decide consistently, and on the record, which of those systems were safe to build and which needed reshaping first. Written as a policy document it produces a PDF; built as a workflow it produces an auditable record: every submission logged with its approval trail, whether or not it was approved.

It follows the Government of Canada's Algorithmic Impact Assessment model under the Directive on Automated Decision-Making, implemented as an executable process. It is the second stage of a funnel: the upstream ethics review decides who needs one.

What gets assessed

Eight steps, 87 scored answers. Everything else in a step is descriptive on purpose: the unscored fields are what make a reviewer read a narrative instead of a number. Under an official-languages obligation, every question and every reference value exists as an English/French pair in the form models rather than behind a translation layer.

Project detailsUnscored: context for the reviewer

Who is asking, and on whose behalf. Routing and accountability metadata. None of it reaches the score.

The governed path

Walked off the BPMN sequence flow, the order is not the one a reader expects. The signatures clear first and the score comes after.

The governed path, in execution order
Registry formaiaaRegistryForm
Initiator resolved at runtime, not routed to a fixed queue.
Manager approvalaiaaRegistryFormApproval
Approve · request edits · reject.
request edits, returns to the registry form
approved · phase = implementation
Director approval
Second signature. Production branch only.
signatures cleared
Assessment record
Written before scoring, approval trail attached.
research phase skips the branch below
Scoring serviceriskRatio
Scored answers reduced to a ratio.
Decision tableaiaa_risk_score · hit policy FIRST
In: phase, riskRatio. Out: riskLevel.
risk_1risk_2risk_3
Trade-off: the criteria are published before anyone submits, so applicants can write toward them. That was accepted: criteria you cannot see before submitting could not be reviewed or contested, which defeats the point of publishing them.
1
Registry formHumanInitiator

The proposing team completes the eight-step assessment. Assignment is dynamic: the process resolves the initiator at runtime rather than routing to a fixed queue.

2
Manager approvalHumanManager

Every submission reaches a manager on a separate approver form: approve, request edits, or reject. Requesting edits routes back to the initiator's form, a real cycle in the process model rather than a dead end.

3
Director approvalDecisionDirector

Systems declared implementation or production need a second signature. Earlier phases skip the branch. The director has the same three outcomes, including sending it back for edits.

4
Create assessmentAutomatedSystem

Once the required signatures clear, the submission is written to a durable assessment record with the approval trail attached. The agency ends up with an inventory of the systems it has assessed.

5
Calculate risk scoresAutomatedSystem

A scoring service reduces the 87 scored answers to a risk ratio. Scoring is a service call rather than form logic, so the same submission always scores the same way. Research-phase submissions skip it.

6
Risk levelDecisionDecision table

A DMN table maps (phase, ratio) to a risk level. The thresholds live in the table, so the people who own the policy revise and re-version them without a deployment.

7
Outcome notificationAutomatedSystem

Terminal states send phase-specific notifications. Pending approvals carry reminder escalations, so an assessment cannot go quiet by being ignored.

How risk is decided

Rules that change on a policy cadence live in a versioned decision table, not in a release. The cost is that the decision logic leaves the code review path: a threshold change ships without a build, so the table itself has to be versioned and its history has to be readable.

Here that table is aiaa_risk_score, using DMN's FIRST hit policy: it takes the declared lifecycle phase and the risk ratio and returns a risk level. The thresholds tighten as the system gets closer to real decisions.

Design & development
≤ 2
Level 1
< 3
Level 2
≥ 3
Level 3
Implementation & production
≤ 1
Level 1
< 2
Level 2
≥ 2
Level 3
A ratio of 2 is the bottom band during design and the top band in production. Same answers, re-judged against what it now costs to be wrong.

Where the decision stays human

The declared lifecycle phase decides how many signatures a submission needs, and whether it is scored at all. Risk classification is a property recorded against an approved assessment; approval authority stays with a person throughout.

Research & explorationManager

Not scored. The process routes around the scoring service. An idea being explored is recorded and signed off, but not risk-classified.

Design & developmentManager

Scored, and classified against the design thresholds.

Implementation & productionManager, then director

Scored against tighter thresholds. Answers that sat at the bottom band during design reach the top band here.

What it demonstrates

The criteria are a form, the risk model is a decision table, the authority is a routed signature, and every assessment leaves a record whether or not it was approved. Those are the mechanisms a regulated institution already runs for consequential decisions, pointed at a newer kind of system.