[Enhancement] OrientationSensor documentation for iOS needs location service note

[skorpsim]

Summary

I had a Xamarin.Forms App that read Accelerometer, Gyroscope and OrientationSensor. On Android everything worked as expected. On iOS only the OrientationSensor did not provide updates. Not a single update was received. Beside the missing updates there was nothing that indicated the error.

The issue was that, in the Xamarin.Essentials source code DataUpdated method of OrientationSensor.ios.watchos.cs was called once and provided a NSError object. This error is not relayed. The error description is:

Error Domain=CMErrorDomain Code=102 "Failed to get true north."
UserInfo={NSLocalizedRecoverySuggestion=Location Services must be available and enabled for System Services >
Compass Calibration., NSLocalizedDescription=Failed to get true north., NSLocalizedFailureReason=Unable to access location.}

Turns out that Location Services must be available and enabled for System Services >
Compass Calibration. This is because CMAttitudeReferenceFrame.XTrueNorthZVertical is dependant on the compass calibration.
All other CMAttitudeReferenceFrame enum values (XArbitraryZVertical, XArbitraryCorrectedZVertical, XMagneticNorthZVertical) are not dependant on compass calibration.

Documentation Changes

[iOS specific OrientationSensor setup]
Location Services must be available and enabled for System Services >
Compass Calibration.

Additional Information

Tested on iOS 14.0.1 (iPhone 8) and iOS 12.4.8 (iPhone 6)

[skorpsim]

Additional Information

The error also occurs when Location Services > System Services >
Compass Calibration was enabled few seconds ago and the calibration is not yet finished.
The calibration process can take some time in my case ~20 seconds.

API Changes

DataUpdated method of OrientationSensor.ios.watchos.cs should throw an specific exception when an NSError is received.

Workaround

Because no Updates will be received by OrientationSensor.ReadingChanged Event, a timeout can be implemented, that triggers an OS specific notification.

or

Implement an iOS specific async method that checks the state of the compass calibration.
Use CLLocationManager.LocationServicesEnabled to check whether all location services are disabled.
Request the calibration state by creating a new instance of CLLocationManager with an own implementation of CLLocationManagerDelegate and call StartUpdatingHeading on the instance.

var manager = new CLLocationManager
{
    Delegate = new MyCLLocationManagerDelegate()
};
manager.StartUpdatingHeading();

To receive the calibration state, override public void UpdatedHeading(CLLocationManager manager, CLHeading newHeading) in the delegate implementation.
newHeading.TrueHeading will be -1 when Compass Calibration is disabled or not yet finished.

[janusw]

Hi Simon,

thanks a lot for reporting this issue (and for your detailed analysis). I might possibly be the one to blame for this bug (at least partially), because I have implemented an OrientationSensor fix for iOS in XE version 1.4.0 (see #981 and #988), which actually introduced the compass into the whole OrientationSensor game (before that fix the compass was apparently not used, and all quaternions were reported relative to an arbitrary direction (the phone orientation at sensor startup).

Incidentally a colleague of mine recently also reported to me that the OrientationSensor is not working on iOS 12. We did not get very far with the debugging yet, but it might actually be the same phenomenon that you describe (although she told me that the location service is enabled, but maybe the compass calibration was not completed).

In that context I have one question: Were you able to observe this failure of the OrientationSensor also on iOS 13 or 14? We only saw it on iOS 12, and I could never reproduce it on 14 (not even when putting the device in flight mode).

Best regards,
Janus

[janusw]

Ok, I get it. I could also reproduce the issue now, by going to Settings -> Privacy -> Location Service -> System Services and turning off "Compass Calibration". After that I get no more updates from the OrientationSensor (even on iOS 14).

[janusw]

The issue was that, in the Xamarin.Essentials source code DataUpdated method of OrientationSensor.ios.watchos.cs was called once and provided a NSError object. This error is not relayed. The error description is:

Error Domain=CMErrorDomain Code=102 "Failed to get true north." UserInfo={NSLocalizedRecoverySuggestion=Location Services must be available and enabled for System Services > Compass Calibration., NSLocalizedDescription=Failed to get true north., NSLocalizedFailureReason=Unable to access location.}

Did some debugging, but I actually can not reproduce this behavior. On iOS 14, with the current XE main branch, I see that DataUpdated is not called at all, thus I also get no NSError object. With which iOS and XE version did you see this?

[skorpsim]

Hi Janus,
my first approach to the issue was testing the xamarin.essentials source code with SourceLinking.
Everything seemed fine except that DataUpdated was NEVER called. I did not fully trust the SourceLinking results because i have never used it before.

I wanted to reimplement the iOS code, according to the apple documentation, and created a xamarin.ios demo project.
Copy/Pasted the xamarin.essentials source code without relevant modifications, found some additional properties and fetched them in a timer.

I wanted to provide a simple demo project, stripped everything out that seemed irrelevant ... and the NSError was gone....
Turn out that fetching a single property, I added for debugging, lead to the one-time call of DataUpdated with the NSError.

This is the added property

public static partial class OrientationSensor
{
        internal static CMDeviceMotion CurrentState =>
            manager?.DeviceMotion;

and the TimerCallback

private void OnTime(object state)
{
    if (OrientationSensor.CurrentState?.Attitude?.Quaternion is CMQuaternion quat)
        Console.WriteLine($"Reading: X: {quat.x}, Y: {quat.y}, Z: {quat.z}, W: {quat.w}");
}

Here is the project. Deactivate the Timer and you won't see the NSError
OriIOS.zip

[janusw]

Hi,

my first approach to the issue was testing the xamarin.essentials source code with SourceLinking.

huh, I never tried that.

I wanted to reimplement the iOS code, according to the apple documentation, and created a xamarin.ios demo project.

In general I guess the best way to debug such problems is to use the XE Samples app as a baseline, because it provides a clean ground state that everone can reproduce (without the additional complexity of an external project and without any code duplication).

Turn out that fetching a single property, I added for debugging, lead to the one-time call of DataUpdated with the NSError.

Interesting. I did not dive into the details of your example project, but one could possibly use this mechanism to detect the problem. It's not the most elegant one, but I haven't found anything better yet.

If you'd like to explore this further, I'd suggest you put it in a small patch on top of the XE repo, using Samples.iOS as demonstrator. That makes it more clear what you need to add for this.

Btw, one other idea that I have tried is to use CMMotionManager.AvailableAttitudeReferenceFrames, but this always seems to return all four frames (including XTrueNorthZVertical).

[janusw]

Maybe we should first come to an agreement about what would be the most reasonable way to deal with this issue.

I can see a couple of options:

1. Documentation (as suggested in your initial report). That certainly would be useful, but it does not fully solve the problem. In particular, if the problem occurs in an app based on XE, the users of that app might not even know that it is based on XE, and thus will not look into the XE docu. In other words: Documentation only helps developers of Xamarin apps, not their users.
2. Abandon XTrueNorthZVertical in the OrientationSensor, and simply use XMagneticNorthZVertical (although it's slightly inferior from the end-user's point of view). Btw, also the XE Compass uses XMagneticNorthZVertical, so the OrientationSensor would then be even more consistent with the compass. In any case, the difference between the two is usually not very large (somewhere around 3 degrees at my location).
3. Try to detect if XTrueNorthZVertical would fail and use XMagneticNorthZVertical as a replacement in that case only. The downside here is that it's not directly clear to the user which reference frame is being used in the end.
4. Try to detect if XTrueNorthZVertical would fail and throw an exception.

(Are there other reasonable options?)
I currently tend to favor the last option, but the question is how one can reliably detect the issue in the first place.

[skorpsim]

but one could possibly use this mechanism to detect the problem.

Yes but the mechanism is not comprehensible.

I also prefer option No.4 and and in my first comment i outlined how you can reliably check for the compass calibration state.
See:

Workaround

Implement an iOS specific async method that checks the state of the compass calibration.
Use CLLocationManager.LocationServicesEnabled to check whether all location services are disabled.
Request the calibration state by creating a new instance of CLLocationManager with an own implementation of CLLocationManagerDelegate and call StartUpdatingHeading on the instance.

var manager = new CLLocationManager
{
    Delegate = new MyCLLocationManagerDelegate()
};
manager.StartUpdatingHeading();

To receive the calibration state, override public void UpdatedHeading(CLLocationManager manager, CLHeading newHeading) in the delegate implementation.
newHeading.TrueHeading will be -1 when Compass Calibration is disabled or not yet finished.

[maRci002]

Android's Sensor.TYPE_ROTATION_VECTOR uses magnetic north, so it is fine to use xMagneticNorthZVertical.