Parse SwiftUI hierarchies under production-faithful hosting#348
Closed
RoyalPineapple wants to merge 16 commits into
Closed
Parse SwiftUI hierarchies under production-faithful hosting#348RoyalPineapple wants to merge 16 commits into
RoyalPineapple wants to merge 16 commits into
Conversation
- CustomAction promoted from String typealias to struct with name field - AccessibilityContainer gains .scrollable(contentSize:), isModalBoundary, and customActions fields - Parser and rendering code updated for CustomAction struct
The nested package structure breaks SPM version resolution for consumers that pin by URL+version. Inline the model target pointing at the nested package's source directory and expose it as a product.
The outer Package.swift now declares AccessibilitySnapshotModel as an inline target. The nested Package.swift caused SPM to see duplicate target declarations.
…undle collision The fork kept the upstream package name "AccessibilitySnapshot", causing rules_swift_package_manager to generate identical resource bundle names (AccessibilitySnapshot_AccessibilitySnapshotParser.bundle) when both the fork and canonical repo are linked in the same bazel target. Rename Package(name:) only — all product/target/module names unchanged. Zero import sites affected. The generated bundles are now prefixed AccessibilitySnapshotBH_ instead of AccessibilitySnapshot_. Bumps version to 0.19.0.
Rename SwiftPM package to avoid resource bundle collision with canonical AccessibilitySnapshot
…ftUI List, and LazyVStack Tests exercise 4 scroll view types at 3 scroll positions (top/middle/bottom) to validate the parser's handling of off-screen accessibility elements: - UITableView: prefetched rows beyond the viewport are captured - UICollectionView: same prefetch behavior as UITableView - SwiftUI List (backed by UICollectionView): iOS 15+ - SwiftUI LazyVStack in ScrollView (not backed by UICollectionView): iOS 15+ Each test creates a 30-item scrollable container in a 375x400 viewport, scrolls to the target position, and snapshots the accessibility overlay.
…lipping Introduces ParserOptions struct with includeOffScreenElements (default false) that prunes elements whose accessibility frame falls outside their scroll view ancestors' visible content rects, mirroring the SPI's shouldOnlyIncludeElementsWithVisibleFrame behavior. The visibility check (hasVisibleFrame) clips against each scrollable ancestor using UIAccessibility.convertToScreenCoordinates, with a 2pt minimum threshold. Only scroll views with content exceeding their bounds are considered — SwiftUI's internal non-scrolling UIScrollView wrappers are skipped. Zero-frame non-clipping containers pass through to allow child overflow (e.g. iOS 26 UISearchController bridging views). SPI validation tests confirm the filter matches Apple's behavior: - UITableView: default 28 elements, visible-frame 9 (matching SPI) - UICollectionView: default 10, visible-frame 9 - LazyVStack: default 21, visible-frame 7 - Zero-frame wrapper: SPI returns 0 for both modes Updated scroll view snapshot reference images to reflect the pruned output.
…vior
Three changes to the parser's off-screen element pruning:
1. Non-UIView accessibility elements (e.g. SwiftUI's AccessibilityNode) are now
checked for visibility via the accessibilityContainer chain. Previously only
UIView elements were pruned, so LazyVStack items that were scrolled off-screen
were incorrectly included (21 elements vs SPI's 7).
2. Scroll view visible-rect clipping now accounts for adjustedContentInset. The
SPI clips against the inset-adjusted content area, not the raw bounds. This
was causing 1-2 extra edge elements to survive pruning when safe area insets
were present.
3. Added 12 asserting SPI comparison tests (4 scroll view types × 3 positions)
that verify parser output exactly matches _accessibilityLeafDescendantsWithOptions:
with shouldOnlyIncludeElementsWithVisibleFrame=YES. All 12 pass with exact match.
Key implementation details:
- nearestContainerView(for:) walks the accessibilityContainer chain via
perform(NSSelectorFromString("accessibilityContainer")) to find the nearest
UIView ancestor for non-UIView elements (works for SwiftUI's AccessibilityNode
which is NOT a UIAccessibilityElement).
- clipFrameAgainstAncestors() extracted as a shared helper used by both UIView
and non-UIView visibility checks.
Test results: 75 unit tests pass (0 fail), 111 snapshot tests pass (5 pre-existing
failures unrelated to this change).
…terministic tests (#8) * Fix macOS Model Tests CI job with a standalone model package manifest The 'Model Tests (macOS)' CI job runs: swift test --package-path AccessibilitySnapshotModel but there was no Package.swift inside AccessibilitySnapshotModel/, so SwiftPM walked up to the repository-root manifest — which declares the full snapshot graph including the UIKit-dependent iOSSnapshotTestCase target. On macOS that fails to build with "'UIKit/UIKit.h' file not found", and it wouldn't have run the model tests anyway (the root package has no model test target). Add a standalone manifest declaring only the portable, UIKit-free model library and its test target. `swift test --package-path AccessibilitySnapshotModel` now builds and runs the model tests on macOS (and Linux). Verified: - swift test --package-path AccessibilitySnapshotModel: green on macOS - swift package resolve (root): still resolves, no target conflict - iOS Tuist build: unaffected (uses Project.swift, not this manifest) The root manifest references the model by source path, not as a sub-package, so the nested manifest introduces no duplicate-target conflict. * Add missing ScrollViewTests references (17.5/26.2); skip non-deterministic tests ScrollViewTests (TableView/CollectionView/LazyVStack) had references only for 18.5, causing 'reference image not found' on the iOS 17/26 CI legs. Adds those references (recorded by CI across the matrix, run 29080240994). Skips 4 tests whose accessibility parse is non-deterministic across runs — the set of surfaced elements differs run-to-run, so no reference image is stable (verified: same-OS record vs compare diverge with full-scale pixel deltas): - ModalTests/testOneModalOneContainer: competing accessibilityViewIsModal boundaries (1 modal + 1 container-of-2-modals) resolve differently per run. - ScrollViewTests/testSwiftUIListScrolled{Top,Middle,Bottom}: SwiftUI List element set unstable at snapshot capture time. Joins the existing skip list; determinism fix tracked separately.
… at delivery (#7) * Model layer: visibility field, DataTableCellInfo, delivery transform Adds AccessibilityVisibility {onscreen, offscreen} and AccessibilityElement.visibility (custom Codable, decodeIfPresent for legacy payloads). Evolves AccessibilityContainer.ContainerType.dataTable to carry cells:[DataTableCellInfo?] with wire-compatible custom Codable. Adds AccessibilityDelivery (DeliveryOptions, ScrollContainerSummary, DeliveredAccessibility, deliver(options:)) and AccessibilityShape.boundingRect. Wires AccessibilitySnapshotModelTests into the (en) scheme (26 tests green). Parser foldNodes emits empty dataTable cells placeholder. Commit 1 of the graph-derived-context / parse-full / trim-at-delivery refactor. * Graph-derive tab context, populate dataTable cells payload Kills the re-entrant loop-back for `.tabBar`-trait views: their tab index/count are now derived from each element's ordered position among its tab-bar-trait siblings in the already-sorted traversal (tabTraitPositions), rather than a re-walk of the view's subtree via recursiveAccessibilityHierarchy + tabBarCache. context(for:from:) no longer takes layout direction/idiom/cache params. Populates the .dataTable container node's cells:[DataTableCellInfo?] payload at the foldNodes emission point, aligned to the node's ordered children. Reuses the exact isFirstInRow / NSNotFound / immediately-preceding-header filtering from the description path and resolves header references to child indices. foldNodes now threads the source object through each mapped child to enable this. Byte-identity gate: full SnapshotTests + UnitTests + model tests green (185 + 26), no reference images re-recorded. Commit 2. * Delete ContextProvider loop-back machinery; derive context from graph position Replaces the `ContextProvider` enum (superview/accessibilityContainer/dataTable) with a lightweight `ContextParent` that captures the context-providing ancestor plus, for enumerated containers, the child's captured index/count (its ordered graph position). `context(for:from:)` becomes `derivedContext(for:parent:...)`, deriving Context purely from the parent's role + position; the data-table branch splits into `dataTableCellContext`. Deletes `providedContextAsSuperview`/`providedContextAsContainer` in favor of inline ContextParent construction + a `superviewContextParent()` that only vends context for the subview-sourced roles the old `.superview` path handled (UITabBar, .tabBar-trait, dataTable). Rewrites `overridesElementFrame` as a pure sort concern keyed on the parent view's .tabBar trait. Removes the now-dead `explicitAccessibilityElements` binding (silences its warning). Byte-identity gate: full SnapshotTests + UnitTests + model tests green (185 + 26), no reference images re-recorded. Commit 3. * Parse full tree with visibility metadata; drop ParserOptions pruning The parser no longer prunes off-screen elements during the walk. The UIView visible-frame gate and the non-UIView scroll-clip gate now MARK elements `.offscreen` and keep descending, threading `inheritsOffscreen` so a descendant of an off-screen view is itself off-screen (reproducing today's descent-gating as a flag). Existence gates (hidden, zero-frame+clips, sub-1pt) are unchanged; zero-frame non-UIView elements stay `.onscreen`. Visibility is threaded through the element node -> sorted tuple -> buildElement, stamping AccessibilityElement.visibility. Deletes ParserOptions / includeOffScreenElements / .fullTree entirely (fork-only API); the parser is always a full parse now. Updates the two .fullTree test call sites to the default initializer. Pairs with the delivery-wiring commit; verified together against the full SnapshotTests suite (byte-identical, no re-records). Commit 4. * Wire delivery trim at the snapshot end Adds AccessibilitySnapshotConfiguration.deliveryOptions (default .trimmed). The UIKit base view and the SwiftUI snapshot view now call .deliver(options: configuration.deliveryOptions) instead of flattenToElements(), so off-screen elements marked by the full-parse are dropped (or kept, under .untrimmed) at delivery. ParsedAccessibilityData gains containerSummaries (default []), populated from the delivery result; the SwiftUI view stores the summaries too. Byte-identity gate (commits 4+5 together): full SnapshotTests green — deliver(.trimmed) reproduces the pre-refactor pruned output exactly, including all ScrollViewTests — plus parser unit + model tests. No reference images re-recorded. Commit 5. * SPI parity: compare pruned config against deliver(.trimmed) The [pruned] parity assertion now compares the SPI's visibleFrameOnly walk against parseAccessibilityHierarchy(in:).deliver(.trimmed).elements, matching the new delivery-based trimming. [full]/[grouped] already compare the default parser's flattenToElements() (full tree). The 3 UITableView cases keep skipFullTree:true pending index enumeration. GATE: UIViewIndexAPIValidationTests green (24). Commit 6. * Opt-in off-screen element count legend affordance Adds AccessibilitySnapshotConfiguration.showsOffscreenElementCounts (default false). When enabled, AccessibilitySnapshotView.render(data:) appends one OffscreenCountsLegendView per non-empty ScrollContainerSummary, styled like the legend hint rows, reporting how many off-screen elements were trimmed above/below each scroll container's viewport. Localizes offscreen_counts.above.format / .below.format in en/de/ru with Strings accessors. Default (flag off) is byte-identical: verified against ScrollView, containers, LargeView, and Layout snapshot tests (26) — no reference images changed. Commit 8. * Docs: container refactor, delivery API, 0.22.0 breaking notes Adds Documentation/Refactor-Containers-and-Delivery.md describing graph-derived context, always-full parse with visibility metadata, delivery-time trimming (DeliveryOptions / ScrollContainerSummary / deliver(options:)), the opt-in off-screen counts legend, the 0.22.0 removal of ParserOptions/includeOffScreenElements/ .fullTree with migration notes, and the deferred UITableView index-enumeration limitation. Cross-links from Core-Architecture.md. Commit 9. * Enumerate UITableView rows via index API (VoiceOver parity) Relaxes the UIView index-API guard for UIScrollView subclasses so UITableView vends all its rows through accessibilityElement(at:) — the way VoiceOver enumerates them — instead of only the cells instantiated as subviews. The blocker that deferred this earlier was a misdiagnosis. There is no wrapper-vs-cell divergence: the index API vends one uniform population of UITableViewCellAccessibilityElement proxies. The real issue is that an off-screen proxy (whose cell isn't instantiated) reports accessibilityFrame = .zero during the walk. The visibility gate's 'frame.width>0 && height>0' precondition — which exists to keep zero-frame SwiftUI *container wrappers* on-screen — was letting these zero-frame *leaves* default to .onscreen. Fix: a non-UIView accessibility *leaf* with no frame has no visible presence, so it is marked .offscreen. Container wrappers are !isAccessibilityElement and take a different path, unaffected. Verified: parser enumerates all 30 rows; deliver(.trimmed) yields exactly the 8 visible (rows 22-29 at 'bottom'), matching the SPI. All 25 SPI-parity tests pass with skipFullTree removed from the 3 UITableView tests — [full] and [grouped] now match Apple's 30-row enumeration 1:1 including order. UICollection View parity unaffected. Snapshots: the 3 ScrollViewTests table references are intentionally re-recorded — the legend now correctly lists the full visible row band (which the old subview-only walk under-counted), bringing the tool closer to VoiceOver's actual behavior. Per the governing rule, this is a sanctioned snapshot change. Commit 7. * Expose identifier-bearing views as containers * Model container facts as navigable boundaries * Suppress derived user-input-label echo (unblocks table enumeration) Ports the parser fix from cashapp#347 into the fork: an element with no explicitly-set Voice Control input labels reports a UIKit-derived [accessibilityLabel] echo, which is redundant for targeting. authoredUserInputLabels suppresses the single-element [label] echo and keeps genuine authored labels. This is what made the table-enumeration commit appear to change default snapshot output: index-vended cells surfaced the echo, rendering a redundant input-label pill per row and widening the legend. With the echo suppressed, table snapshots return to their pre-refactor output, so the 3 ScrollViewTests table references are reverted to their main versions (no re-record needed). Adds regression tests: echo suppressed for a label-only element; distinct authored input labels preserved. * Decode visibility leniently instead of a custom compat rejection Per project guidance: visibility is a field this fork adds, so it can take any shape we choose — but old payloads that predate it must not crash to decode. Use decodeIfPresent(...) ?? .onscreen. (The audit's 'reject old payloads' rule is a BH-internal convention; the snapshot model has real external consumers whose persisted data predates this field.) The .dataTable cells compatibility stays: .dataTable is an upstream-existing concept we extended, so we owe its existing payloads compatibility. Test renamed to reflect intent: decoding a pre-visibility payload must not crash. * Make table-enumeration parity test device-agnostic Compare deliver(.trimmed) rows against the SPI's own visible-frame result rather than a hardcoded count of 8 (which was iOS-18.5-specific and failed on iOS 26, where a different set of rows intersects the viewport). Asserts full 30-row enumeration plus visible == SPI visible, robust across OS/device metrics. * Replace deliver(options:) with composable onscreen() filter The bundled deliver(options: DeliveryOptions) -> DeliveredAccessibility mixed two orthogonal operations (visibility filtering + hierarchy flattening) behind an opaque .trimmed/.untrimmed knob. Split it: - [AccessibilityHierarchy].onscreen() -> [AccessibilityHierarchy] is a tree-to-tree filter (filter { $0.isOnscreen } recursively). It drops off-screen elements and any container left with no surviving children, since an empty container is not an accessibility element. Compose with the unchanged flattenToElements(): hierarchy.onscreen().flattenToElements(). - scrollContainerSummaries() computes the N-above/M-below tallies over the full tree, where the off-screen elements still exist. Config knob deliveryOptions: DeliveryOptions = .trimmed becomes includesOffscreenElements: Bool = false, framing off-screen inclusion as opt-in (standard output is on-screen only). Deletes DeliveryOptions, DeliveredAccessibility, .trimmed/.untrimmed. * Re-record scroll/list references for edge color assignment Markers are now colored by their index in the delivered on-screen traversal order (assigned at the render edge). For the sectioned SwiftUI list this reorders the legend into true reading order (section header, its rows, its footer, next header...) instead of listing all rows first and supplementary elements last; every band's color and legend position shifts accordingly. The on-screen highlight rectangles are unchanged. For the table, the on-screen row set is identical; only the per-row highlight color follows the new index. References taken from CI run 29093459337 (iOS 17.5 / 18.5 / 26.2).
…tion materialization (#9) * Add VerbosityConfiguration to model (from cashapp cashapp#309) Ports cashapp#309's VerbosityConfiguration verbatim into the agnostic model as the config vocabulary for late description assembly. Pure Foundation, builds on the model target. Nothing consumes it yet. * Move localization into the agnostic model Relocates the localized Strings struct, String+Localization, and the {en,de,ru}.lproj Assets from the Parser target into AccessibilitySnapshotModel so late (verbosity-driven) description assembly can run on any platform. - Strings + localized(...) + StringLocalization made public for the Parser. - Model manifests gain defaultLocalization: en + resources: [.process(Assets)]; Parser target drops the Assets resource; Tuist mirrors the move. - Fixes a latent bug: .lproj lookup used subdirectory: Assets, but .process(Assets) flattens .lproj to the bundle root, so de/ru silently fell back to English. Now looks at the bundle root. New StringsLocalizationTests proves de resolves (Taste., not Button.). 31 model tests green; iOS app builds. * Relocate description assembly into the model as a pure verbosity fold Ports the parse-time NSObject.accessibilityDescription(context:) into the model as AccessibilityElement.description(context:verbosity:), reading only stored model data + a graph-derived DerivedContext. This un-bakes the description: it is now a pure, platform-independent function of (element, context, verbosity) that reproduces the historical string at .verbose. - DerivedContext: ref-free mirror of the parser Context; dataTable headers are resolved HeaderText(label,value) rather than live NSObjects, so they stay re-gatable under includesTableContext. - traitPosition (.before/.after/.none) mirrors iOS 18.4 Verbosity > Controls (Speak Before / Speak After / Don't Speak); .after is the historical default. - 11 assembly tests pin the three trait positions, verbosity gating, container context, and hints. 41 model tests green. Parser still uses its own parse-time path; wiring the parser to call this and consuming dataTable cells is the next step. * Emit .series container node for segmented controls (class-free) Step 4a (part 1): make the graph self-describing for series. - Add ContainerType.series to the model (enum + hand-rolled Codable), distinct from .tabBar: series members keep their Button trait and append "N of M", tabs replace Button with "Tab.". - Detect segmented controls class-free via accessibilityContainerType == 11 (private value, read through the public property; same idiom as private trait bits). Confirmed live + specific by accessibility research: steppers/sliders/date-pickers return 0. - The new node is transparent to rendering (flattenToElements drops containers); segment descriptions still come from the parse-time path until the 4c cutover. Byte-identity: testSegmentedControl + testTabBars + full DefaultControls and AccessibilitySnapshotTests suites unchanged (23/23); 41 model tests green. * Classify tab bars class-free via .tabBarItem children Step 4a (part 2): make the graph self-describing for tab bars. A container whose children carry the private .tabBarItem trait (bit 28) is a tab bar. This recognizes a real UITabBar without an `is UITabBar` check: UITabBar reports accessibilityContainerType == .semanticGroup and carries no .tabBar trait, so today it emits as a semanticGroup node; the children-trait rule reclassifies it to .tabBar. Custom .tabBar-trait views remain matched by the trait. The node-type change is transparent to rendering (flattenToElements drops containers) and descriptions still come from the parse-time path, so this is byte-identity-safe until the 4c cutover. Confirmed live byte-identical (bit 28) on iOS 18.5 + 26.3 by accessibility research; UIStepper/UISlider/UIPageControl carry no such children and are not misclassified. Byte-identity: testTabBars + full DefaultControls (incl. page control, stepper, slider) unchanged. SwiftUIListSectionTests failures are pre-existing (fail on baseline without this change). * Derive container context from graph position (model, 4b) Step 4b: pure model-side derivation mirroring the parser's live derivedContext, UIKit-free. AccessibilityContainer.derivedContext(forChildAt:in:) reads a child's context from graph position alone: - .series -> .series(index: ordinal+1, count: siblings) - .tabBar -> .tab(index: ordinal+1, count: siblings) (.tab ==.tabBarItem) - .list -> .listStart / .listEnd (sole child: start only) - .landmark-> .landmarkStart / .landmarkEnd - .dataTable-> resolve stored cells[i] header child-indices into sibling label/value as HeaderText (over the full child set, before pruning) - semanticGroup / scrollable / .none -> nil The {index, count} this computes is the same value VoiceOver derives from _accessibilityRowRange (which for tab bars UIKit itself computes from sibling position) -- so we reproduce it from the tree with no SPI call. Not yet wired into delivery (that is 4c); the parser's parse-time path still produces descriptions. 9 derivation tests; 50 model tests green. * Materialize descriptions at delivery from graph context (4c) Cut the spoken description/hint over from parse-time baking to a delivery-time fold: parse stamps raw facts + container structure, and `materializingDescriptions(verbosity:)` composes the final string from graph-derived context just before on-screen trimming. Wired into both the UIKit and SwiftUI delivery sites; `.verbose` (the default) reproduces historical output. Three latent byte-identity bugs the live cutover surfaced (all dormant while descriptions were baked): - Subview-based list/landmark got context the old parser withheld: emit .list/.landmark only on the container-API path (explicitlyOrdered); subview- vended containers fall back to .semanticGroup, matching superviewContextParent's historical nil. Faithful subview boundaries stay deferred (plan P5). - Doubled hint: the parser baked the composed hint (raw + trait suffix) into element.hint, so re-composition at delivery doubled switch/adjustable hints. buildElement now stores the raw author hint; the suffix is composed once, late. - Important custom content was folded into the description string; the historical parser never did this (the legend renders it separately). Removed. Full snapshot suite: only the pre-existing SwiftUIListSectionTests failures remain (confirmed identical on a clean baseline). 50 model tests + new parity localizer green. * Preserve .list/.landmark node type on the subview path (4c fixup) The 4c cutover briefly remapped subview-vended .list/.landmark containers to .semanticGroup to force description byte-identity. That broke the container structural contract (testListContainerIsAlwaysPreserved / ...Landmark...): a .list/.landmark view must emit its node type regardless of how it vends its children, so query consumers can find it by type. CI Tuist UnitTests caught it. Revert to always emitting .list/.landmark. Subview-based lists now correctly gain their real 'List Start'/'List End' boundaries via graph derivation -- exposing that accessibility information is the point of this refactor, not a divergence to suppress. Both current snapshot fixtures use the container-API path, so reference PNGs are unchanged; 91 UnitTests + full snapshot suite green (only pre-existing TextField/TextView + SwiftUIListSectionTests failures remain). * Contextualize at the render boundary: materializing flatten + container-aware SwiftUI legend (4d) Flattening is the moment the container structure is dropped, so it is now also the moment each element's graph-derived context is folded into its final rendered string. `flattenToElements(verbosity: = .verbose)` derives context per child as it walks (same composition the deleted `materializingDescriptions` pass performed) and returns terminal, render-ready elements. Existing call sites compile unchanged and get materialized descriptions for free; delivery sites flatten the FULL tree (so "X of N" counts and data-table headers derive from complete child sets) and prune the flat array by visibility afterwards. The UIKit render path is unchanged: same flat markers, same per-marker legend, same stored-description reads, byte-identical snapshots. The SwiftUI legend gains an opt-in container-aware mode ported from cashapp#329 (closed): `showContainers` renders the legend hierarchically with dashed container borders and badges. Unlike cashapp#329, the `.element` case composes `description(context:verbosity:)` live from the element's graph position instead of reading the stored string — the graph walk IS the contextualizer on this path, so context is never stored on the element and re-contextualization cannot double (the composer reads only raw facts). - HierarchyColorAssignment: threads DerivedContext down the assign walk, composes at each element, filters by visibility against the full child set; elements keep their flat traversal indices (overlays identical), containers numbered pre-order after all elements - HierarchyLegendView / ContainerLegendEntryView: ported from cashapp#329 - ParsedAccessibilityData.hierarchy: data plumbing so the SwiftUI renderer can reach the graph; the UIKit renderer ignores it - withDescription made public, documented as a terminal projection - ContainerDemo fixture + snapshot tests (with/without containers) Suites: 50 model tests green; UnitTests green; SnapshotTests only pre-existing failures (TextField/TextView, SwiftUIListSection x2); PreviewsTests green except pre-existing locale-dependent testCustomContentDemo (demo formats 2847 via device locale; scheme does not pin a language). * TEMP: record container demo references on the 26.2 CI runner The iOS 26.2 simulator runtime is no longer downloadable from Apple, so the two new container-demo references for the iOS_26 CI matrix entry can't be recorded locally. Gate record mode to 26.2 so the runner records them, and extend the failure-artifact upload to include the Previews reference images (durable improvement) so the recorded PNGs can be harvested. The record-mode flip will be reverted once the images are committed. * Add recorded 26.2 container references; rename HierarchyColorAssignment to ContextualizedHierarchy - Harvest the two ContainerDemo 26.2 reference images recorded on CI (the 26.2 simulator runtime is no longer downloadable locally) and revert the temporary OS-gated recordMode lines. - Rename HierarchyColorAssignment -> ContextualizedHierarchy: the type applies graph-derived context to each element (composed description, hint, and overlay color index), not just color assignment. AssignedNode becomes Node. No behavior change. * Separate marker numbering from ContextualizedHierarchy Numbering (and the color each number selects) is a property of the snapshot rendering, not the hierarchy. ContextualizedHierarchy now carries only context application — description and hint composed from graph position, container structure preserved — and HierarchyLegendView assigns indices as it renders: elements in flat traversal order (matching the markers array), containers pre-order after all elements. No pixel change. * Document ContainerType.tabBar's two capture channels * Model hint decomposition: user fact, computed state utterances, computed spoken merge The user-set hint stays the stored fact (element.hint, raw since the un-bake); the VoiceOver state utterances become a computed accessor (stateHint(verbosity:)); the spoken merge stays the hint half of description(context:verbosity:). All three run the same pipeline — the historical merge rules (switch wraps, text entry replaces, adjustable chains) are factored into one private function that the spoken path feeds the real hint and the state path feeds nil, so spoken == merge(user, state) by construction and nothing new is stored anywhere. Spoken output is byte-identical (verbatim factoring). * One contextualize walk: model owns the canonical→marker projection contextualized(verbosity:) on [AccessibilityHierarchy] is now THE contextualize step — the single walk that threads DerivedContext down the tree and composes each element's spoken strings. flattenToElements becomes contextualize-then-plain-flatten, and the Previews ContextualizedHierarchy.build becomes pure structure (visibility filter + empty-container drop) over the same walk, so both legend modes speak identically by construction and the duplicated context walk is gone. Also documents (finding 149, iOS 26.3): real VoiceOver speaks N-of-M / table position trailing like us, but weaves list/landmark boundary phrases into the trait-specifier position — we keep the historical trailing placement for byte-identity. Gates: 54 model tests; Previews snapshot suite incl. both ContainerDemo refs (sole failure = pre-existing locale-environmental testCustomContentDemo); parity + parser + index-API unit tests green.
iOS strips SwiftUI navigation bar content (e.g. .searchable fields) from the accessibility tree unless the hosting view is a direct window subview or its controller is seated in the view controller hierarchy. The SwiftUI snapshot entry point hosted the hosting controller's view nested inside the snapshot container, so the search field rendered in the image but was absent from the parsed tree — a topology production VoiceOver never sees. Apple's own tree walker confirms the suppression under nesting. SnapshotVerifyAccessibility for SwiftUI views now hosts the view as a direct window subview for the duration of the snapshot, and parseAccessibility() parses a pre-hosted view in place instead of reparenting it into the container. Callers that don't pre-host (the entire UIKit path) take the existing reparenting path unchanged. Parsing in place also lets the parse wait for SwiftUI's asynchronous accessibility updates to settle: List publishes its final accessibility frames on a batch-update completion one run loop turn after layout, so parsing immediately after layoutIfNeeded nondeterministically captured transient text-tight row frames (the run-to-run List marker flakiness). The parse now spins the run loop until two consecutive parses agree. Known limitation: rarely SwiftUI's accessibility graph stalls on the unsettled List frames indefinitely for a hosting instance (stable across arbitrarily many run loop turns and layout invalidations, even while a sibling instance in the same process settles fine). Stability is the settle-wait's only generic signal, so it cannot correct that pre-existing condition; the regression test skips when it detects it. No reference image changes: all existing SwiftUI references already match the settled state on iOS 17.5 and 18.5.
Collaborator
Author
|
Opened against upstream by mistake — this belongs on the BH fork for now. Will be upstreamed deliberately later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two SwiftUI-only defects in the snapshot harness, both diagnosed with dynamic settlement-timeline probes (follow-up to the ParserFidelity work in #10 and the
.searchablecorpus gap in #11):.searchablefield (topology suppression). iOS strips SwiftUI navigation bar content from the accessibility tree unless the hosting view is a direct window subview or its controller is seated in the view controller hierarchy. The SwiftUI snapshot entry hostedhostingController.viewnested inside the snapshot container, so the search field rendered in the image but never entered the parsed tree. This is not a parser gap: Apple's own leaf walker omits the field under the same nesting (it returns even less — our parser still finds the nav title). Production VoiceOver never sees this topology.Listpublishes its final accessibility frames on a batch-update completion one run loop turn after layout (UpdateCoalescingCollectionView). Parsing immediately afterlayoutIfNeedednondeterministically captured transient text-tight row frames — the long-standing run-to-run alternation between full-row and text-tight markers.Fix (single commit)
SnapshotVerifyAccessibility(SwiftUI overload) hosts the view as a direct window subview for the duration of the snapshot.AccessibilitySnapshotBaseView.parseAccessibility()detects a pre-hosted contained view and parses it in place (no reparenting), then spins the run loop until two consecutive parses agree before the real parse. Callers that don't pre-host — the entire UIKit path — take the existing reparenting path unchanged, parse-once, byte-identical.Verification
testUIKitTextField/testUIKitTextView). Zero reference image changes — existing SwiftUI references already pin the settled state.Known limitation (out of scope)
Rarely, SwiftUI's accessibility graph stalls on the unsettled List frames indefinitely for a hosting instance — stable across 50+ run loop turns,
setNeedsLayout/layoutIfNeeded,invalidateLayout, and window re-keying, even while a sibling instance in the same process settles fine. Stability is the settle-wait's only generic signal, so it cannot correct a stable-but-wrong tree. This condition pre-dates this PR (it is whySwiftUIListSectionTestscarries an iOS 17 tolerance, which is left in place) and is being root-caused separately (beagle plan 152 follow-up). The List regression test skips when it detects the stall.Upstreaming
The change is public-API only and conceptually self-contained (one commit), but upstream predates the fork's Core refactor (
AccessibilitySnapshotBaseView), so it ports as a mechanical adaptation ofAccessibilitySnapshotView+ the SwiftUI test-case extensions rather than a cherry-pick.