-
Notifications
You must be signed in to change notification settings - Fork 8.8k
Description
CC @lhecker
Compared the following two versions side-by-side:
- commit e80aadd on Release config (immediately previous to PR Rewrite the MSAA/UIA integration into conhost #19344)
- WT Canary OpenConsole on 9/23/2025 v1.25.2651.0 (should point to commit 4600c47 aka PR Rewrite the MSAA/UIA integration into conhost #19344)
Tested with Narrator and NVDA. NVDA works fine. The following issues were found with Narrator.
Scenario 1: inputting text
Scenario: type "12345" slowly
Expected: screen reader should say "1 2 3 4 5" as it writes each character
Actual: screen reader says "1 1 2 2 3 3 4 4 5 5"
Looks like there's a "local echo". In Windows Terminal, this would normally be caused by the screen reader reading the input character then the newly drawn character. It might be possible that a similar thing is now happening here. We got around it in #12358 by checking if the output text matched the input text, and suppressing that notification.
Scenario 2: moving the cursor
Scenario: with the cursor at the right end of "12345", slowly press the left button
Expected: screen reader should read each character as the cursor is on top of it
Actual: screen reader is silent
NOTE: it looks like the screen reader normally reads the character specifically when the cursor blinks. Definitely bizarre.
Scenario 3: deleting text
Scenario: with the cursor at the right end of "12345", slowly press the backspace button
Expected: screen reader should say "5 4 3 2 1" as it deletes each character
Actual: screen reader is silent
Debugging Tips
AccEvent
Docs: https://learn.microsoft.com/en-us/windows/win32/winauto/accessible-event-watcher
Path: "C:\Program Files (x86)\Windows Kits\10\bin\10.0.26100.0\x64\accevent.exe"
You can use this to take a look at any accessibility events that are being dispatched.
The main ones to look out for are...
- Text_TextChanged: should be raised whenever text is changed (added, deleted, etc.)
- Text_TextSelectionChanged: raised whenever the selection of text changes, or whenever the insertion point (caret) moves among the text.
It looks like the second one isn't being dispatched at all, so that'll probably go a long way to fixing the issues above.
Telemetry
We have some telemetry set up for UIA here:
terminal/src/types/UiaTracing.cpp
Line 13 in 4600c47
// tl:{e7ebce59-2161-572d-b263-2f16a6afb9e5} |
I'm totally blanking on the app I used to hook into this guid, but you should be able to register the guid and see exactly what is running under the hood. Feel free to update UiaTracing.cpp and UiaTextRangeBase.cpp accordingly if you want more information than what's provided.
Other helpful docs and tips
https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-implementingtextandtextrange
It may be worth running OpenConsole as a Release config, I've had trouble sometimes where the Debug config is too slow and the screen reader times out.