helix-rum-bundler

Process RUM events into bundles by domain/time. Provides API to read bundles.

Bundler

0. RUM events logged to S3 in files
1. triggered by cron (every 10min)
2. read files from logs bucket in batches of N
3. sort RUM event into bundle
4. append or write to RUM bundle
5. move raw file to processed location

Related Resources

• EventBridge schedule
• bucket (/helix-rum-bundles)
  • contains bundled RUM events in the format: /{domain}/{year}/{month}/{date}/{utc_hour}.json
  • each date/ directory contains a "bundle manifest" to track sessions
• bucket (/helix-rum-logs)
  • /raw/: raw event log location, each file in this folder is a single unprocessed RUM event
  • /processed/: processed event location, unbundled, eg. /{domain}/{year}/{month}/{date}/{utc_hour}/{date}-{id}.log
• bucket (helix-rum-users)
  • /orgs/: org data
  • /domains/: { org -> orgkey } maps

Bundle Manifest

Contains information needed to efficiently relate new RUM events to an existing session.

{
  "sessions": {
    "id--path": {
      "hour": 1
    },
    "J3Ed2--/some/path": {
      "hour": 2
    }
  }
}

API

bundles

requires authorization, domainkey

• GET /bundles/{domain}/{year}/${month}/{date}/{hour}
• GET /bundles/{domain}/{year}/${month}/{date}
• GET /bundles/{domain}/{year}/${month}

// response
{
  "rumBundles": [
    {
      "id": "foo",
      "time": "2024-01-01T01:02:03+00:00",
      "timeSlot": "2024-01-01T01:00:00+00:00",
      "url": "https://www.example.com/my/path",
      "userAgent": "desktop",
      "weight": 10,
      "events": [
        {
          "checkpoint": "viewmedia",
          "timeDelta": 123, // ms since timeSlot
          "target": "https://www.example.com/my/image.png",
          "source": ".my-block"
        },
        {
          "checkpoint": "loadresource",
          "timeDelta": 123,
          "source": "https://www.example.com/nav.plain.html",
          "target": "1"
        },
        {
          "checkpoint": "cwv"
        },
        {
          "checkpoint": "cwv-lcp",
          "value": 1.1
        }
      ]
    }
  ]
}

domainkey

requires authorization, membership in allowlist

• GET /domainkey/{domain}
• POST /domainkey/{domain}
• DELETE /domainkey/{domain}

orgs

requires orgadmin authorization

• GET /org/{id}
• POST /orgs/{id}/key

requires superuser authorization

• GET /orgs
• POST /orgs
• POST /orgs/{id}
• DELETE /orgs/{id}/domains/{domain}
• GET /orgs/{id}/key
• PUT /orgs/{id}/key