AI Scribe Bias Testing Clinic Assessment Tool

1

Assessment Result

2

Purpose

This assessment will help clinicians understand their readiness to identify and mitigate potential bias in AI Scribe outputs. The goal is to support safer and more equitable use of AI scribes across patient populations.

Instructions

For each question, select either: Yes or No
A progress bar will guide you through the sections.

3

1. Have you or your clinic staff intentionally discussed or received training on AI-related bias, fairness, or responsible AI scribe use?* This field is required.Understanding sources of bias helps prevent inequitable documentation.

YESNO

4

2. Is there a designated person or process in place for evaluating AI scribe output, including recognizing and managing bias?* This field is required.A clear oversight role or process supports accountability and ethical practice.

YESNO

5

3. Does your clinic intentionally create opportunities (e.g meetings, policies, updates) to raise awareness about AI use and how to recognize and address potential risks?* This field is required.Regularly building awareness ensures responsible use of AI remains part of everyday practice.

YESNO

6

4. Do you intentionally compare AI scribe performance across different patient demographics? * This field is required.Equity checks help ensure consistent performance across diverse populations.

YESNO

7

5. Do you intentionally include rare, complex, or non-routine cases when evaluating AI scribe performance?* This field is required.Testing edge cases help identify blind spots and performance limitations.

YESNO

8

6. Has your AI scribe vendor provided information on how they test for accuracy and bias?* This field is required.Transparent vendors help users understand limitations and anticipate risk.

YESNO

9

7. Do you have a defined process to report AI errors or biased output back to the vendor?* This field is required.Feedback loops drive system improvement and strengthen trust.

YESNO

10

8. Do you routinely compare AI-generated notes to non-AI clinician-generated notes? * This field is required.Comparisons help detect variations, omissions, and potential bias.

YESNO

11

9. Do you review AI-generated notes for tone, neutrality, and inclusive language? * This field is required.Monitoring language ensures that documentation remains respectful and unbiased.

YESNO

12

10. Do you always review and edit AI-generated notes before final sign-off?* This field is required.Active human oversight mitigates automation bias.

YESNO

13

11. Do you track how often AI-generated notes require correction? * This field is required.Tracking helps identify potential issues such as performance drift and helps to minimize over-reliance.

YESNO

14

12. Do you routinely use reflection prompts when an AI generated note is completed such as “Would this note change if the patient were different?” * This field is required.Reflection ensures fairness, reduces automation bias and improves clinical judgment.

YESNO

15

13. Do you maintain an audit log for biased output or persistent inaccuracies? * This field is required.Audit records support continuity, accountability, and system improvement.

YESNO

16

14. Is AI bias testing integrated into your clinic’s quality improvement or periodic review processes? * This field is required.Regular checks embed fairness into clinical operations.

YESNO

17

Your Assessment is now complete!

Before you view your report, please take a moment to reflect and tell us whether this assessment was useful to you and your practice. Your feedback will help us improve future versions of this tool.

This section is optional. Select Next to continue.

18

As a result of completing this assessment, are you considering any changes to how you use or monitor your AI scribe?

Yes

Maybe

No

19

What, if anything do you plan to do differently?This question is optional. Please do not include any identifying information, and any requests for further support can be directed to PSP@doctorsofbc.ca.

20

Total Score

21

Report