How to ignore HTTPS certificate verification for HTTP nodes?

[nooooom]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.11.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When using HTTP node to access https protocol, the error CERTIFICATE_VERIFY_FAILED is displayed

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot[bot]]

To ignore HTTPS certificate verification for HTTP nodes in a self-hosted Dify setup, you can modify the SSL context in your application to bypass certificate verification. Here's a general approach you can take in Python:

1. Disable SSL Verification: You can disable SSL verification by creating an unverified SSL context. This is generally not recommended for production environments due to security risks, but it can be useful for testing purposes.

   import ssl
   import openai
   
   # Create an unverified SSL context
   ssl_context = ssl._create_unverified_context()
   
   # Configure the OpenAI client to use this SSL context
   openai.api_key = "your-openai-api-key"
   openai.api_base = "https://your-dify-instance-url"
   openai.api_client = openai.OpenAIClient(ssl_context=ssl_context)
   
   # Example API call
   response = openai.Completion.create(
       engine="davinci",
       prompt="Hello, world!",
       max_tokens=5
   )
   print(response.choices[0].text.strip())

2. Environment Variable: Alternatively, you can set an environment variable to ignore SSL verification. This method depends on the specific libraries and tools you are using.

Remember, bypassing SSL verification can expose your application to man-in-the-middle attacks, so it should only be used in controlled environments where security is not a concern.

To continue talking to Dosu, mention @dosu.

[cainiaochaocai]

Update the api/core/helper/ssrf_proxy.py file to include the verify option in the proxy configuration. The updated code should look like this:

if dify_config.SSRF_PROXY_ALL_URL:
    with httpx.Client(proxy=dify_config.SSRF_PROXY_ALL_URL, verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)
elif dify_config.SSRF_PROXY_HTTP_URL and dify_config.SSRF_PROXY_HTTPS_URL:
    proxy_mounts = {
        "http://": httpx.HTTPTransport(proxy=dify_config.SSRF_PROXY_HTTP_URL),
        "https://": httpx.HTTPTransport(proxy=dify_config.SSRF_PROXY_HTTPS_URL, verify=HTTP_REQUEST_NODE_SSL_VERIFY),
    }
    with httpx.Client(mounts=proxy_mounts, verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)
else:
    with httpx.Client(verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)

The original code only configured the verify option in the httpx.Client, but when deployed using docker-compose, the .env file typically includes proxy settings. As a result, the code often falls into the elif dify_config.SSRF_PROXY_HTTP_URL and dify_config.SSRF_PROXY_HTTPS_URL: branch.

Therefore, it’s necessary to include the verify option when configuring the proxies as well.

The original code was:

if dify_config.SSRF_PROXY_ALL_URL:
    with httpx.Client(proxy=dify_config.SSRF_PROXY_ALL_URL, verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)
elif dify_config.SSRF_PROXY_HTTP_URL and dify_config.SSRF_PROXY_HTTPS_URL:
    proxy_mounts = {
        "http://": httpx.HTTPTransport(proxy=dify_config.SSRF_PROXY_HTTP_URL),
        "https://": httpx.HTTPTransport(proxy=dify_config.SSRF_PROXY_HTTPS_URL),
    }
    with httpx.Client(mounts=proxy_mounts, verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)
else:
    with httpx.Client(verify=HTTP_REQUEST_NODE_SSL_VERIFY) as client:
        response = client.request(method=method, url=url, **kwargs)

Additionally, update the docker-compose.yaml file by adding the following volume configuration to the api service:

- ./volumes/app/ssrf_proxy.py:/app/api/core/helper/ssrf_proxy.py

• This setup is based on Dify version 1.1.3.