AI Evidence Analyzer FAQs

1. What is TrustArc AI Evidence Analyzer? 
   
   AI Evidence Analyzer is designed to automate and enhance compliance management. It assigns an AI-generated control effectiveness score based on the relevance and quality of the submitted evidence (e.g., the CCPA Privacy Policy, Data Processing Agreements). This submitted evidence includes the descriptions of Accountability Mechanisms (AMs). 
   
   It evaluates the evidence and assigns an AI Evidence Score (Excellent, Good, Fair, Poor, Evidence Required, Evidence Not Required, No URL/Attachments, Not Applicable, Unaddressed) based on control descriptions, uploaded evidence quality, and compliance requirements.
   
   As part of the evaluation, the AI Evidence Analyzer identifies the extent to which relevant controls are addressed and recommends ways to improve control effectiveness and close compliance gaps.
    
2. What’s a control?
   
   Controls are the administrative, technical, or physical safeguards that an organization must implement to comply with its requirements. These can be requirements of laws, regulations, and industry-recognized frameworks. Within PrivacyCentral, questions, tasks, and accountability mechanisms are mapped to controls. 
    

3. How does AI Evidence Analyzer help with compliance? 
   
   AI Evidence Analyzer enhances governance and accountability by assessing the quality and relevance of your evidence. It includes:

   • A score for control effectiveness
   • Highlights open compliance gaps
   • Provides clear recommendations for the improvement of evidence and AMs

   It helps privacy teams focus on weaker areas of compliance with a law, regulation, or industry framework and take recommended actions to allocate resources to closing compliance gaps.
    

4. What underlying AI model does AI Evidence Analyzer use?
   
   AI Analyzer uses a variety of AI models, currently GPT-4o mini. This leading model is recognized for its 
   superior textual intelligence and multimodal reasoning capabilities. GPT-4o Mini scores better at reasoning than other options, like Gemini Flash and Claude Haiku. 
   
   The model only uses TrustArc’s control descriptions and legal citations to analyze your evidence and does not pull in external information. 
    
5. Does AI Evidence Analyzer save or share my information with OpenAI for training?
   
   No, your information is not used by OpenAI for training or saved with OpenAI. Your evidence documentation is only saved within your TrustArc application. For input context, evidence information might be sent to the LLM, but it is not shared with OpenAI. 
    

6. How does the AI Evidence Score work? 
   
   The AI Evidence Score evaluates submitted evidence against control descriptions and legal citations. The AI Evidence Analyzer determines which control sections are addressed, highlights fulfilled and unfulfilled areas, generates a numeric score and qualitative rating, and makes recommendations for improvement.
   
   The scoring is as follows: 

   • Excellent: The evidence provided comprehensively covers the evidentiary requirements in the legislation. There are no significant gaps in coverage.
   • Good: The overall guideline coverage is moderate, with some key areas addressed but notable gaps remaining.
   • Fair: The evidence provided partially addresses the regulation requirements and the associated control description, but notable gaps need to be addressed for full compliance.
   • Poor: The guidelines' overall coverage is poor, with significant gaps in compliance with the regulation and the associated controls.
   • Evidence Required: The initial state of the system before any interaction. At this point, no evidence of a control or accountability mechanism has been uploaded.
   • Evidence Not Required: This control doesn’t require you to attach any evidence.
   • No URL/Attachments: Evidence is required for this control, and nothing has been attached so far.
   • Not Applicable: Control does not apply.
   • Unaddressed: You have provided evidence, but it doesn’t meet the guidelines' requirements, leaving significant gaps in coverage.

   The scoring scale helps users easily understand the strength of their evidence and make informed decisions on prioritizing privacy program activities for compliance readiness. 
    

7. How does AI Evidence Analyzer process image files (e.g. uploaded evidence)?
   
   AI Evidence Analyzer uses GPT-4o mini to analyze the content in an uploaded image. The model uses computer vision/OCR capabilities to generate a description of the image content, which is then fed to the LLM model for further processing.
    
8. Which regulations does AI Evidence Analyzer support? 
   
   AI Evidence Analyzer reviews evidence against all over 20,000 controls and 140+ standards available within PrivacyCentral. View PrivacyCentral’s comprehensive library here.
    
9. Is AI Evidence Analyzer suitable for all industries? 
   
   Yes, AI Evidence Analyzer is ideal for any organization with complex regulatory requirements, particularly those in Healthcare, Financial Services, and Technology. It is also ideal for any organization that handles high volumes of personal or sensitive data and wants to assess its compliance readiness with multiple global laws and standards.
    
10. How do I get started with AI Evidence Analyzer? 
    
    Getting started is easy—schedule a demo with our team here, or contact your Account Manager to see AI Evidence Analyzer in action and explore how it can specifically benefit your organization's compliance management.
    
    If you are already a PrivacyCentral user, you can turn it on from the Assessment Settings, then toggle “Evaluate Evidence Score using AI” on individual laws with PrivacyCentral.
    
    
    
    
     
11. What is an Accountability Mechanism?
    
    Accountability mechanisms are the structures, processes, and policies required to ensure compliance with a law. To demonstrate the presence of accountability mechanisms, you need to upload evidence to the AI Evidence Analyzer. Examples of evidence include a Data Protection Officer (DPO) job description and employment contract, where the Accountability Mechanisms for these pieces of evidence would be DPO, security and privacy responsibilities, and privacy roles.
     
12. Which AI model does AI Evidence Analyzer use?
    
    AI Evidence Analyzer uses GPT-4o mini to craft the answer. Please check this page to learn more about 
    our model use.
     
13. Is company data used to train the AI model? If so, can we opt out?
    
    No client data is shared to train the model.
     
14. Is the tool off by default or on by default?
    
    Off by default.
     
15. TrustArc AI supports Human-in-the-Loop review; however, are there any measures in place to prevent harm or bias by the AI?
    
    We have a feedback loop to determine accuracy, and then, based on the feedback, we adjust prompts, creativity, and other thresholds.
     
16. Does the vendor perform regular model updates, and are there internal standards for model training?
    
    The AI Evidence Analyzer utilizes AI models that the TrustArc team deems best for user needs (in this 
    case, it is GPT 4o-mini). We don't hyper-train the model, but we can change the internal prompts we give to the model.
     
17. Are AI outputs labeled, and can users override the AI functionality?
    
    AI outputs are labelled. Users can't override the outputs, and the outputs don't affect the overall efficiency/effectiveness measures. However, users can update the Effectiveness Score for these individual controls.
     
18. Is data provenance logged or tracked?
    
    We don't track any data or its label. We do track the counts of thumbs up and thumbs down as feedback.
     

For more information about AI Evidence Analyzer, visit our technical documentation.