Add mean F3 finality distance from head in terms of number of epochs to public dashboards

[masih]

Update the dashboard to report average distance from relative head in form of number of epochs over 1 week, 1 month and 6 months.

We have this metric already on grafana for the latest instance.
This can also be extracted purely from the certificate chain.

[masih]

We need to know the epoch at which F3 finalized to then measure the relative distance. For this maybe we need to update the observer to also record the finality certs along with the timestamp at which they were observed: #745

[masih]

Another way to approximate the same "insight" is to report the average or rather percentiles of the length of chain finalized. The longer the chain the slower F3 is progressing relative to filecoin chain growth. In an ideal state the length of chain should not be more than 2 tipsets.

[Kubuxu]

https://grafana.f3.eng.filoz.org/d/edsu1k5s7gtfkb/f3-passive-testing?orgId=1&refresh=1m&from=1744968942260&to=1744979742260&viewPanel=75

Created a metric here, percentile over time of distance form latest with different SLAs and time windows (we can maybe make time windows larger although IDK how nicely performant these queries will be)

[masih]

Discussed in standup:
We might need to run influxdb for pesisting data from extended period of times

[BigLep]

2025-04-24 conversation: we can resolve this by spending money by maintaining data for 6 months.
We can also make it run faster by making it an hourly query that is precomputed ("recording rules").