Sync Thread network preference on Matter commissioning from frontend, if necessary

[jpelgrom]

Summary

This PR tries to improve the user experience for commissioning of Matter over Thread devices, by actively syncing a dataset if necessary.

Background

In #4150, the full Thread credential sync was removed from the start of Matter commissioning because of the slow GMS API. However, for users with a Thread network only known to HA (like those who added a ZBT-2, or Apple network synced from an iOS device!), this means Matter over Thread device commissioning would fail unless a full sync was manually triggered from companion app settings.

After the removal, the frontend started sending information about the preferred Thread network, if any, in the commissioning payload. Instead of doing a full sync every time, this allows us to sync the device only if necessary by checking if the device knows these credentials.

Note that the GMS API unfortunately hasn't seen any improvement and we still cannot specify which Thread network to use, so the check/sync remains best effort, instead of being sure this Thread network is used for commissioning like the frontend's intention.

Code in this PR

If the frontend commissioning payload includes Thread credentials, the app checks if the device prefers the network for these credentials (we can do this without asking for permissions or showing additional UI).

• If there are no credentials in the payload, or the device prefers these credentials: the Matter commissioning flow starts.
• If there are credentials: if the app has never synced we do a full sync once, once synced the Matter commissioning flow starts.

In a flowchart this looks like:

flowchart LR
    A(Frontend starts Matter commissioning) --> B{Payload with Thread credentials?}
    B -- No --> C(Start Matter commissioning flow)
    B -- Yes --> D{Thread network from credentials preferred by device?}
    D -- Yes --> C
    D -- No --> E{Thread synced at least once?}
    E -- No --> F[Perform full sync]
    F --> C
    E -- Yes --> C

Loading

While testing, asking the GMS API whether the dataset is preferred still takes almost exactly 10 seconds most of the time, but not always - I suspect it is waiting for a timeout somewhere. That means this change still slows down starting commissioning, but only if core knows about Thread and less than a full sync.

Checklist

• New or updated tests have been added to cover the changes following the testing guidelines.
• The code follows the project's code style and best_practices.
• The changes have been thoroughly tested, and edge cases have been considered.
• Changes are backward compatible whenever feasible. Any breaking changes are documented in the changelog for users and/or in the code for developers depending on the relevance.

Screenshots

Link to pull request in documentation repositories

n/a

Any other notes

Discussion on Discord that started this PR.

This PR is still draft while I want to

• test a bit more with removing credentials and when to set/unset the has synced boolean
• polish exception handling without changing all the code

[bgoncal]

Note that the GMS API unfortunately hasn't seen any improvement and we still cannot specify which Thread network to use, so the check/sync remains best effort, instead of being sure this Thread network is used for commissioning like the frontend's intention.

So this means that when comissioning a new matter over thread device, the preferred thread network selected in Home Assistant will not be respected right?

What would happen in my case where I have one HA thread network, one Aqara and one Apple, which one it would decide to use? (Considering all of them have their credentials saved on my Google Pixel/Google services)

[jpelgrom]

So this means that when comissioning a new matter over thread device, the preferred thread network selected in Home Assistant will not be respected right?

It may or may not be, depending on which network was added first.

What would happen in my case where I have one HA thread network, one Aqara and one Apple, which one it would decide to use? (Considering all of them have their credentials saved on my Google Pixel/Google services)

Right now, it seems that whatever app adds the first credentials will become the preferred. Apps cannot influence this, and the user also cannot change this.

All of this isn't new behavior. It already applies whenever you commission a Matter device today and manually sync Thread from the companion app settings.

[bgoncal]

@jpelgrom I understand, thats just frustrating that they dont open up the APIs to at least be able to define to which thread network to add the device to. Hopefully Matter improvements will bring the so called "single network" where networks will merge into one

[TimoPtr]

Could you split a bit this function? It has too many layer and I think having smaller functions with proper names would make it easier to follow what's going on.

[jpelgrom]

I've split it into a couple of smaller but still logically grouped functions, with a description for each function. Hopefully this is a good middle ground between extremely tiny and the original.

[TimoPtr]

Any reason to create a new scope?

[jpelgrom]

Short answer: because I reused the previously removed code.

Longer answer: because I want a SupervisorJob to ensure any failures here don't impact the rest of the rest of WebViewPresenterImpl- mainScope uses a normal Job. See: #3634. We could also change mainScope to use a SupervisorJob if that's a better solution in your opinion.

[jpelgrom]

After testing various scenarios, it seems to work mostly as expected but last minute I thought of the need to handle the scenario:

• core previously preferred dataset x, the app synced and the device also prefers it
• core now prefers dataset y instead of x, the device should be updated to match it

This can be included in the 'fast' check by triggering sync if

• there has never been a sync (✅)
• there has been a sync (✅) which resulted in adding a dataset for x, which is different from y which core is now preferring (TODO)

So this PR will stay in draft for a little longer, while I work on that change (most likely Wednesday) as well as a test for each yes/no of the flowchart.

Improving the quality of the Matter/Thread components itself, like more specific error handling, is a larger task that I want to keep outside the scope of this PR. The specific error class is only because I'm making something new, not changing existing code with it for now.