Skip to content

Added exposed API key custom metric #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Added exposed API key custom metric #165

wants to merge 1 commit into from

Conversation

vsdaan
Copy link

@vsdaan vsdaan commented Jun 13, 2025

Added custom metric to check for exposed API keys in both inline scripts and external sources.

Test websites:

@github-actions GitHub Actions
Copy link

https://almanac.httparchive.org/en/2022/

WPT result details

Changed custom metrics values:

{
  "_exposed_keys": []
}
https://example.com

WPT result details

Changed custom metrics values:

{
  "_exposed_keys": []
}
https://daanvs.be/apitest

WPT result details

Changed custom metrics values:

{
  "_exposed_keys": [
    "gemini",
    "openai"
  ]
}

@max-ostapenko max-ostapenko requested a review from Copilot June 13, 2025 10:10
Copilot
Copilot AI reviewed Jun 13, 2025
View reviewed changes
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

max-ostapenko
max-ostapenko requested changes Jun 13, 2025
View reviewed changes
dist/exposed_keys.js
Comment on lines +64 to +65
for (let i = 0; i < key_providers.length; i++) {
const regex = new RegExp(key_regex[i], 'g');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid array index issues(if these 2 lists are closely related) can we put them in the same object and iterate over object items?

dist/exposed_keys.js
const key_regex = ['\\b(aio\\_[a-zA-Z0-9]{28})\\b', '\\b(sk-ant-(?:admin01|api03)-[\\w\\-]{93}AA)\\b', '\\b(apify\\_api\\_[a-zA-Z-0-9]{36})\\b', '\\b(v1\\.0-[A-Za-z0-9-]{171})\\b', '\\b(CFPAT-[a-zA-Z0-9_\\-]{43})\\b', '\\b([a-z0-9-]+(?:\\.[a-z0-9-]+)*\\.(cloud\\.databricks\\.com|gcp\\.databricks\\.com|azuredatabricks\\.net))\\b', '\\b(web\\_[0-9a-z]{32})\\b', '\\b((?:dop|doo|dor)_v1_[a-f0-9]{64})\\b', '(https:\\/\\/discord\\.com\\/api\\/webhooks\\/[0-9]{18,19}\\/[0-9a-zA-Z-]{68})', '\\b(ey[a-zA-Z0-9]{34}.ey[a-zA-Z0-9]{154}.[a-zA-Z0-9_-]{43})\\b', '\\b(dp\\.pt\\.[a-zA-Z0-9]{43})\\b', '\\b(API_KEY[0-9A-Z]{32})\\b', '\\b(flb_live_[0-9a-zA-Z]{20})\\b', '\\b(shltm_[0-9a-zA-Z-_]{40})', '\\b(FLWSECK-[0-9a-z]{32}-X)\\b', '\\b(fio-u-[0-9a-zA-Z_-]{64})\\b', '\\bftp://[\\S]{3,50}:([\\S]{3,50})@[-.%\\w\\/:]+\\b', '\\{[^{]+auth_provider_x509_cert_url[^}]+\\}', '\\{[^{]+client_secret[^}]+\\}', '\\b((?:master-|account-)[0-9A-Za-z]{20})\\b', '\\b(live_[0-9A-Za-z\\_\\-]{40}[ "\'\\r\\n]{1})', '\\b(glc_eyJ[A-Za-z0-9+\\/=]{60,160})', '\\b(glsa_[0-9a-zA-Z_]{41})\\b', '\\b(gsk_[a-zA-Z0-9]{52})\\b', '\\b(?:hf_|api_org_)[a-zA-Z0-9]{34}\\b', '\\b(s-s4t2(?:ud|af)-[a-f0-9]{64})\\b', 'jdbc:[\\w]{3,10}:[^\\s"\'<>,(){}[\\]&]{10,512}', '\\b(pk_[a-zA-Z0-9]{34})\\b', '\\b((?:api|sdk)-[a-z0-9]{8}-[a-z0-9]{4}-4[a-z0-9]{3}-[a-z0-9]{4}-[a-z0-9]{12})\\b', '\\b(lin_api_[0-9A-Za-z]{40})\\b', '\\b(pk\\.[a-zA-Z-0-9]{32})\\b', '[0-9a-f]{32}-us[0-9]{1,2}', '(https:\\/\\/[a-zA-Z-0-9]+\\.webhook\\.office\\.com\\/webhookb2\\/[a-zA-Z-0-9]{8}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{12}\\@[a-zA-Z-0-9]{8}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{12}\\/IncomingWebhook\\/[a-zA-Z-0-9]{32}\\/[a-zA-Z-0-9]{8}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{4}-[a-zA-Z-0-9]{12})', '\\b(NF\\-[a-zA-Z0-9]{32})\\b', '\\b(secret_[A-Za-z0-9]{43})\\b', '(npm_[0-9a-zA-Z]{36})', '\\b(nvapi-[a-zA-Z0-9_-]{64})\\b', '\\b(sk-[a-zA-Z0-9_-]+T3BlbkFJ[a-zA-Z0-9_-]+)\\b', '\\b(ak_live_[a-zA-Z0-9]{30})\\b', '\\b(sk\\_[a-z]{1,}\\_[A-Za-z0-9]{40})\\b', '\\b(phx_[a-zA-Z0-9_]{43})\\b', '\\b(PMAK-[a-zA-Z-0-9]{59})\\b', '\\b(pnu_[a-zA-Z0-9]{36})\\b', '-----\\s*?BEGIN[ A-Z0-9_-]*?PRIVATE KEY\\s*?-----[\\s\\S]*?----\\s*?END[ A-Z0-9_-]*? PRIVATE KEY\\s*?-----', '\\b(sub-c-[0-9a-z]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12})\\b', '\\b(pul-[a-z0-9]{40})\\b', '\\b(?:amqps?):\\/\\/[\\S]{3,50}:([\\S]{3,50})@[-.%\\w\\/:]+\\b', '\\b(ramp_id_[a-zA-Z0-9]{40})\\b', '\\brzp_live_[A-Za-z0-9]{14}\\b', '(rdme_[a-z0-9]{70})', '\\b(ey[a-zA-Z0-9-._]{153}.ey[a-zA-Z0-9-._]{916,1000})\\b', '\\bredi[s]{1,2}://[\\S]{3,50}:([\\S]{3,50})@[-.%\\w\\/:]+\\b', '\\b(r8_[0-9A-Za-z-_]{37})\\b', '\\b(rh-api-[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})\\b', '\\b(rubygems_[a-zA0-9]{48})\\b', '\\bSG\\.[\\w\\-]{20,24}\\.[\\w\\-]{39,50}\\b', '\\b(xkeysib\\-[A-Za-z0-9_-]{81})\\b', '\\b(shppa_|shpat_)([0-9A-Fa-f]{32})\\b', '\\b(slk_[a-f0-9]{64})\\b', '(?:sandbox-)?sq0i[a-z]{2}-[0-9A-Za-z_-]{22,43}', '\\b(sq0idp-[0-9A-Za-z]{22})\\b', '\\b(sbp_[a-z0-9]{40})\\b', '\\btskey-[a-z]+-[0-9A-Za-z_]+-[0-9A-Za-z_]+\\b', '\\b([A-Za-z0-9]{14}.atlasv1.[A-Za-z0-9]{67})\\b', '(https://[\\w-]+\\.tines\\.com/webhook/[a-z0-9]{32}/[a-z0-9]{32})', '\\bthog-key-[0-9a-f]{16}\\b', '\\bAC[0-9a-f]{32}\\b', '\\b(BBFF-[0-9a-zA-Z]{30})\\b', '\\bhttps?:\\/\\/[\\w!#$%&()*+,\\-./;<=>?@[\\\\\\]^_{|}~]{0,50}:([\\w!#$%&()*+,\\-./:;<=>?[\\\\\\]^_{|}~]{3,50})@[a-zA-Z0-9.-]+(?:\\.[a-zA-Z]{2,})?(?::\\d{1,5})?[\\w/]+\\b', '\\b(VF\\.(?:(?:DM|WS)\\.)?[a-fA-F0-9]{24}\\.[a-zA-Z0-9]{16})\\b', '\\b(xai-[0-9a-zA-Z_]{80})\\b', '(https:\\/\\/hooks\\.zapier\\.com\\/hooks\\/catch\\/[A-Za-z0-9\\/]{16})', '\\b(1000\\.[a-f0-9]{32}\\.[a-f0-9]{32})\\b']
const key_providers = ['adafruitio', 'anthropic', 'apify', 'cloudflarecakey', 'contentfulpersonalaccesstoken', 'databrickstoken', 'dfuse', 'digitaloceanv2', 'discordwebhook', 'documo', 'doppler', 'finage', 'fleetbase', 'flexport', 'flutterwave', 'frameio', 'ftp', 'gcp', 'gcpapplicationdefaultcredentials', 'gemini', 'gocardless', 'grafana', 'grafanaserviceaccount', 'groq', 'huggingface', 'intra42', 'jdbc', 'klaviyo', 'launchdarkly', 'linearapi', 'locationiq', 'mailchimp', 'microsoftteamswebhook', 'nightfall', 'notion', 'npmtokenv2', 'nvapi', 'openai', 'pagarme', 'paystack', 'posthog', 'postman', 'prefect', 'privatekey', 'pubnubsubscriptionkey', 'pulumi', 'rabbitmq', 'ramp', 'razorpay', 'readme', 'reallysimplesystems', 'redis', 'replicate', 'robinhoodcrypto', 'rubygems', 'sendgrid', 'sendinbluev2', 'shopify', 'sourcegraphcody', 'squareapp', 'squareup', 'supabasetoken', 'tailscale', 'terraformcloudpersonaltoken', 'tineswebhook', 'trufflehogenterprise', 'twilio', 'ubidots', 'uri', 'voiceflow', 'xai', 'zapierwebhook', 'zohocrm']

const scripts = Array.from(document.scripts);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have all the script response bodies available in global WPT_BODIES variable already.
Will it work matches lookup?

Example: https://docs.webpagetest.org/custom-metrics/examples/blocking-scripts/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

Copilot code review Copilot Copilot left review comments

@max-ostapenko max-ostapenko max-ostapenko requested changes

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vsdaan @max-ostapenko