Search

AI Risk Technical Documentation

Overview

Risk Profile uses TrustArc's AI Risk Algorithm to calculate control effectiveness and residual risk directly from your assessment responses. To take advantage of this, your AI risk assessment needs to be prepared with the correct labeling before it can generate dynamic risk reporting.

This article covers how to prepare your AI risk assessment for technical validation, the rules TrustArc uses to validate your labels, and the steps to replace a TrustArc AI Risk Assessment with your own custom assessment in Risk Profile.

What you can do
✓Label question responses so Risk Profile can calculate control effectiveness and residual AI risk
✓Tag questions so the AI Risk Report auto-populates the AI system name, description, and organizational role
✓Replace a TrustArc AI Risk Assessment with your own custom assessment in Risk Profile
Prerequisites
✓Active TrustArc account with access to Risk Profile and Assessment Manager
✓An assessment published and completed at least once in Assessment Manager
Preparing Your Assessment

To ensure your assessment can leverage the full capabilities of Risk Profile — including TrustArc's AI Risk Algorithm and dynamic risk reporting — you must first prepare your AI risk assessment for technical validation.

TrustArc uses labels on question responses to calculate control effectiveness and residual risk, and breaks those calculations out by Trustworthy AI categories that align with TrustArc's Responsible AI Framework. Question tags and role labels are used to pull information into the AI Risk Report.

Labels must be correctly implemented on your assessment to ensure the risk and control effectiveness calculations in your risk reports are accurate, and to populate the AI Risk Report fields. TrustArc's validation and technical testing verifies that the AI risk algorithm and AI risk calculations are working properly, giving you confidence that everything works as expected before you roll your configured AI risk assessments out across your organization.

📋 Note: The Validation Rules section below outlines exactly which labels and tags are required on which questions and question responses.
Validation Rules

Your assessment, and any associated child assessments, are all validated and tested at the same time.

Response Tags
•For a question's response to be included in the report, it must contain a specific response tag. Two questions require this. The response to the first question is used to fill in the name of the AI system being assessed in the AI Risk Report — the (AI_System_Name) tag is required for the AI system name to appear in the report's AI System Name field.
Example question tagged with AI_System_Name so the AI system name populates in the AI Risk Report

Figure 1.0

The response to the second question provides a description of the AI system for the AI Risk Report. The (AI_System_Description) tag is required for the AI System Description field to be populated in the report.

Example question tagged with AI_System_Description so the AI system description populates in the AI Risk Report

Figure 2.0

⚠️ Warning: These tags must be configured exactly as shown in Figures 1.0 and 2.0, and each can only be used once in the assessment. If a tag is missing, the corresponding field in the report will not be populated.
Role Labels
•To fill in the Role field of the AI Risk Report, questions about the organization's role in AI management must include two labels: the specific role and the organizational role, as shown in Figure 3.0. If no role labels are present in the assessment, the Role field in the report remains empty. Each role label should only be used once in the assessment.
Example question labeled with an AI management role and organizational role

Figure 3.0

Control Effectiveness and Trustworthy AI Category Labels
•Questions used to determine control effectiveness must have both a Control Effectiveness label and a Trustworthy AI Category label on each response, to calculate control effectiveness and residual risk. Additional labels on a response are fine — they don't affect the calculations.
•Questions used only to gather information about the AI system being assessed — or to determine a fact and whether a control applies — do not require labels.
Example question response with paired Control Effectiveness and Trustworthy AI Category labels

Figure 4.0

📋 Example: Your assessment might ask a Yes/No question to determine whether an organization's AI processing is high risk. This question is used to determine a fact, so it does not require Control Effectiveness or Trustworthy AI Category labels.
Rules and Error Messages

The table below lists the rules your AI risk assessment is tested against, and the error messages returned if a rule is implemented incorrectly.

Rule Error message returned
Supported Control Effectiveness values are 0, 1, 2, 3, or 4. Any Control Effectiveness label with a value greater than 4 will return an error. Control Effectiveness cannot be greater than 4
Each answer can have only zero or one Control Effectiveness label. More than one on a single response will return an error. Only one Control Effectiveness is allowed per answer
Every Control Effectiveness label must be paired with a Trustworthy AI Category label. A Control Effectiveness label without one will return an error. Control Effectiveness exists but Trustworthy AI Category is undefined
The presence of two or more Trustworthy AI Category labels on a single response will return an error. Only one Trustworthy AI Category is allowed per answer
All responses for a single question must share the same Trustworthy AI Category label. A mismatched response will return an error. Only one Trustworthy AI Category is allowed per question
The assessment must contain at least one question with a defined Control Effectiveness and Trustworthy AI Category. If none exist, an error is returned. Template has no risk label defined

Table 1.0

Valid AI Management Role Labels

These are the AI management role labels recognized by TrustArc's platform to populate the Role field in the AI Risk Report. Each is paired with an "Organization Role" label, as described above.

•"Provider" + "Organization Role"
•"Deployer" + "Organization Role"
•"Distributor" + "Organization Role"
•"Importer" + "Organization Role"
•"Other" + "Organization Role"

Table 2.0

Valid Control Effectiveness Labels

These are the Control Effectiveness labels recognized by TrustArc's platform intelligence to calculate control effectiveness and risk when generating your risk report.

•"Control Effectiveness = 0"
•"Control Effectiveness = 1"
•"Control Effectiveness = 2"
•"Control Effectiveness = 3"
•"Control Effectiveness = 4"

Table 3.0

Valid Trustworthy AI Category Labels

These are the Trustworthy AI Category labels recognized by TrustArc's platform intelligence to calculate control effectiveness by control category and data transfer risk when generating your risk report. They are paired with Control Effectiveness labels, as shown in Figure 4.0.

•"Valid and Reliable"
•"Explainable and Interapertable"
•"Accountable and Transparent"
•"Privacy Enhanced"
•"Fair with Risk of Harmful Bias Managed"
•"Safe"
•"Secure and Resilient"

Table 4.0

Replacing a TrustArc AI Risk Assessment

To replace a TrustArc AI Risk Assessment with one of your own, follow these steps:

1
Publish the assessment you want to use as a replacement, and complete it at least once in Assessment Manager. This makes it available in the right-hand column of the Risk Assessment Configuration page's Add Template screen, where you select which TrustArc AI risk assessments to replace.
📋 Note: Publishing a replacement assessment does not impact your organization's Risk Profile calculations. The assessment created in Assessment Manager can be deleted later if needed.
2
On the Add Template screen, indicate which TrustArc AI risk assessment(s) you want to replace with your own.
3
TrustArc conducts testing to confirm your labels are properly implemented and that the risk and control effectiveness calculations work correctly in the report.
4
TrustArc notifies you once your assessment is approved. It will then appear on the Start Assessment window within the Risk & Assessments tab, under the AI Risk tab, of a Hub business process or system record.
Start Assessment window showing an approved custom AI risk assessment under the AI Risk tab

Figure 6.0

Removing a Custom Assessment

If you choose to remove your approved custom assessment at a later time, click the trash can icon next to it. A confirmation message appears asking whether you'd like to delete the assessment template you used to replace a default TrustArc AI risk assessment.

Delete confirmation dialog for a custom assessment used to replace a default AI risk assessment

Figure 7.0

TrustArc Trust Center  ·  Preparing Your AI Risk Assessment for Risk Profile  ·  support.trustarc.com