Skip to content

root/storages: fix punycode handling for s3 presigned generation #14706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

dominic-r
Copy link
Contributor

@dominic-r dominic-r commented May 26, 2025

What

This PR fixes an issue with S3 presigned URL generation when using custom domains that contain Unicode characters. The previous implementation was incorrectly handling punycode domains, resulting in "SignatureDoesNotMatch" errors when users attempted to access uploaded files.

Problem

When using S3 storage with a custom domain containing Unicode characters (e.g., "ニャー"), the signature calculation was broken because the old implementation used simple string replacement (url.replace(root_url, custom_url)) which didn't properly handle URL structures with punycode domains.

Closes

Closes #13463

Acceptance Criteria

  • Storage-related tests pass:
authentik/root/tests/test_storages.py::TestS3Storage::test_url_without_custom_domain PASSED                                                                                         [ 12%]
authentik/root/tests/test_storages.py::TestS3Storage::test_endpoint_restored_after_exception PASSED                                                                                 [ 25%]
authentik/root/tests/test_storages.py::TestS3Storage::test_url_with_custom_domain_no_querystring_auth PASSED                                                                        [ 37%]
authentik/root/tests/test_storages.py::TestS3Storage::test_url_with_unicode_custom_domain PASSED                                                                                    [ 50%]
authentik/root/tests/test_storages.py::TestS3Storage::test_url_with_different_scheme PASSED                                                                                         [ 62%]
authentik/root/tests/test_storages.py::TestS3Storage::test_url_with_special_characters_in_path PASSED                                                                               [ 75%]
authentik/root/tests/test_storages.py::TestS3Storage::test_url_with_punycode_domain PASSED                                                                                          [ 87%]
authentik/root/tests/test_storages.py::TestS3Storage::test_bucket_name_in_path_handling PASSED                                                                                      [100%]
  • Lint checks for Black and Ruff pass
  • Ideally, the issue creator could confirm resolution.

Details

REPLACE ME


Checklist

  • Local tests pass (ak test authentik/)
  • The code has been formatted (make lint-fix)

If an API change has been made

  • The API schema has been updated (make gen-build)

If changes to the frontend have been made

  • The code has been formatted (make web)

If applicable

  • The documentation has been updated
  • The documentation has been formatted (make website)

@dominic-r dominic-r requested a review from a team as a code owner May 26, 2025 22:43
Copy link

netlify bot commented May 26, 2025

Deploy Preview for authentik-docs canceled.

Name Link
🔨 Latest commit 9800ea9
🔍 Latest deploy log https://app.netlify.com/projects/authentik-docs/deploys/685557966a04280008f9825b

Copy link

netlify bot commented May 26, 2025

Deploy Preview for authentik-storybook canceled.

Name Link
🔨 Latest commit 9800ea9
🔍 Latest deploy log https://app.netlify.com/projects/authentik-storybook/deploys/685557956a04280008f98259

Copy link

codecov bot commented May 26, 2025

Codecov Report

Attention: Patch coverage is 98.58491% with 3 lines in your changes missing coverage. Please review.

Project coverage is 92.75%. Comparing base (a640866) to head (9800ea9).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
authentik/root/tests/test_storages.py 98.44% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #14706      +/-   ##
==========================================
- Coverage   92.80%   92.75%   -0.05%     
==========================================
  Files         822      823       +1     
  Lines       42384    42588     +204     
==========================================
+ Hits        39333    39503     +170     
- Misses       3051     3085      +34     
Flag Coverage Δ
e2e 47.68% <0.00%> (-0.01%) ⬇️
integration 24.35% <0.00%> (-0.01%) ⬇️
unit 90.66% <98.58%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dominic-r dominic-r force-pushed the sdko/s3-punycode-fix branch from 86f2bea to 7de7e88 Compare May 26, 2025 23:00
@rissson rissson requested a review from BeryJu May 27, 2025 11:21
Copy link

netlify bot commented Jun 20, 2025

Deploy Preview for authentik-integrations canceled.

Name Link
🔨 Latest commit 9800ea9
🔍 Latest deploy log https://app.netlify.com/projects/authentik-integrations/deploys/685557956ab1710008806921

try:
# If custom domain is set, configure the endpoint URL
if self.custom_domain:
scheme = "https" if self.secure_urls else "http"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fails on this line with:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/risson/goauthentik/authentik/main/.venv/lib/python3.13/site-packages/django/db/models/fields/files.py", line 70, in url
    return self.storage.url(self.name)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/risson/goauthentik/authentik/main/authentik/root/storages.py", line 101, in url
    scheme = "https" if self.secure_urls else "http"
                        ^^^^^^^^^^^^^^^^
AttributeError: 'S3Storage' object has no attribute 'secure_urls'

I was doing application.meta_icon.url

@rissson rissson self-requested a review June 20, 2025 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

s3 custom_url makes signature verification fail
2 participants