aci_rest module giving 503 Service unavailable error (DCNE-327)

[prasanthvelunny78]

Not sure if I am doing something wrong but I am getting <urlopen error Tunnel connection failed: 503 Service Unavailable>" error when using aci_rest module. All other module works fine (epg, bridge domain, switch profile etc). I am using python:3.12-slim image in pipeline. The same scripts works on another ansible machine in the same network. I checked the proxy, network connection etc and all looks good. I am also getting successful response when i am using curl to that url in the pipeline script.

Just wondering if you have seen this error before ?

[akinross]

From your description it is hard for me to say anything meaningful, could you provide your installed python environment and ansible installation details, your ansible playbook with configuration, and logs ( set output level to debug and run playbook with high verbosity level ).

Furthermore do you experience the issue only in CI or also as standalone task?

[prasanthvelunny78]

Apologies, Please find the below details and I also attached the pipeline jobs.

Python version 3.12.9
Ansible version : 2.18.1

As you can see in the below pipeline configuration, all of those tasks completed successfully. Including Ansible dry run, NDI precheck, Deployment and NDI post check. When I introduce third job (- Backup) in the pipeline to take ACI backup it gives me an error saying 503 Service available (Added the ansible backup script as well)

Same backup script works on ansible server which is on the same network as Gitlab server.

So basically, I want to perform a ACI backup before pushing the configuration using aci_rest module.

If you require more details please let me know

Many Thanks

Pipeline tasks

stages:

• Dry-run
• Precheck
• Backup
• Deploy
• Postcheck

ansible_dry_run:
stage: Dry-run
image: alpine/ansible:2.18.1
variables:
ANSIBLE_CONFIG: ansible.cfg
script:

- ansible-playbook -i hosts.yaml create_epg_v3.yml --check  --extra-vars  "password=$password Input_file=epg_input.csv"

artifacts:
when: always
paths:
- /builds/root/aci-epg/output/output.json

only:

• test-with-variable
• manual

NDI-Pre-change-validation:
stage: Precheck
image:
name: python:3.12-slim
variables:
ANSIBLE_CONFIG: ansible.cfg
https_proxy: "http://myproxy.com"
no_proxy: "localhost,10.0.0.0/8"
script:

- python3 -m venv /venv  # Create a virtual environment
- source /venv/bin/activate  # Activate virtual environment
- pip install requests_toolbelt  # Install requests_toolbe
- pip install ansible
- ansible --version

- ansible-galaxy collection install cisco.nd
- ANSIBLE_PYTHON_INTERPRETER=/venv/bin/python3
- ansible-playbook -i hosts.yaml ndi_pcv.yml --extra-vars  "password=$password Input_file=epg_input.csv"
needs:
- ansible_dry_run
only:
  - test-with-variable

Production_Deployment:
stage: Deploy
image:
name: alpine/ansible:2.18.1
variables:
ANSIBLE_CONFIG: ansible.cfg
script:

- pwd
- ls -l
- ansible --version
- ansible-playbook -i hosts.yaml create_epg_v3.yml --extra-vars  "password=$password Input_file=epg_input.csv"

needs:
- NDI-Pre-change-validation

only:

• test-with-variable

NDI-Post-change-validation:
stage: Postcheck
image:
name: python:3.12-slim
variables:
ANSIBLE_CONFIG: ansible.cfg
https_proxy: "http://myproxy.com"
no_proxy: "localhost,10.0.0.0/8"
script:

- python3 -m venv /venv  # Create a virtual environment
- source /venv/bin/activate  # Activate virtual environment
- pip install requests_toolbelt  # Install requests_toolbelt
- pip install ansible

- ansible-galaxy collection install cisco.nd
- ANSIBLE_PYTHON_INTERPRETER=/venv/bin/python3
- ansible-playbook -i hosts.yaml ndi_post_change.yml --extra-vars --extra-vars  "password=$password Input_file=epg_input.csv"

needs:
- Production_Deployment

only:
- test-with-variable

================================================
Ansible backup script

tasks:
- name: Create Fabric backup
tags: BACKUP
aci_rest:
host: "{{ host }}"
username: "{{ username }}"
password: "{{ password }}"
validate_certs: False
path: /api/mo/uni.json
method: post
content: |
{
"configExportP": {
"attributes": {
"dn": "uni/fabric/configexp-Postman",
"name": "Postman",
"snapshot": "true",
"targetDn": "",
"adminSt": "triggered",
"rn": "configexp-Postman",
"status": "created,modified",
"descr": "{{ Change_number }}-Fabric-Pre-change"
},
"children":[]
}
}

[shrsr]

@prasanthvelunny78
Can you please post the following:

1. Command used to run the ansible backup script?
2. EPG tasks in Ansible?
3. The complete snippet of the error message?
4. Can you re-check the proxy config used when running the Ansible backup?

[prasanthvelunny78]

Hi There, Please find the details below. I edited username and password due to our internal security reasons. Other than that there is no changes in the logs. 1. ACI API request. - name: Prompt for host hosts: localhost connection: local become: false gather_facts: false tasks: - name: Create Fabric backup tags: BACKUP aci_rest: host: IP username: username password: "{{ password }}" validate_certs: False path: /api/mo/uni.json method: post content: | { "configExportP": { "attributes": { "dn": "uni/fabric/configexp-Postman", "name": "Postman", "snapshot": "true", "targetDn": "", "adminSt": "triggered", "rn": "configexp-Postman", "status": "created,modified", "descr": "Fabric-Pre-change" }, "children":[] } } 2. I am currently testing only with the Backup. All other modules are working. - name: Create EPG aci_epg: host: "{{ Fabric }}" username: username password: "{{ password }}" tenant: "{{ item.Tenant }}" ap: "{{ item.AP }}" epg: "{{ item.EPG }}" bd: "{{ item.Bridge_domain }}" description: "{{ item.Description }}" state: "{{ item.State }}" validate_certs: no output_lev 3. Error message $ ansible-playbook -i hosts.yaml backup.yml -vvvv --check --extra-vars "password=$password Input_file=epg_input.csv" ansible-playbook [core 2.18.4] config file = /builds/root/aci-epg/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /venv/lib/python3.12/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /venv/bin/ansible-playbook python version = 3.12.9 (main, Mar 17 2025, 23:25:08) [GCC 12.2.0] (/venv/bin/python3) jinja version = 3.1.6 libyaml = True Using /builds/root/aci-epg/ansible.cfg as config file setting up inventory plugins Loading collection ansible.builtin from host_list declined parsing /builds/root/aci-epg/hosts.yaml as it did not pass its verify_file() method script declined parsing /builds/root/aci-epg/hosts.yaml as it did not pass its verify_file() method Parsed /builds/root/aci-epg/hosts.yaml inventory source with yaml plugin redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest Loading collection cisco.aci from /venv/lib/python3.12/site-packages/ansible_collections/cisco/aci Loading callback plugin default of type stdout, v2.0 from /venv/lib/python3.12/site-packages/ansible/plugins/callback/default.py Skipping callback 'default', as we already have a stdout callback. Skipping callback 'minimal', as we already have a stdout callback. Skipping callback 'oneline', as we already have a stdout callback. PLAYBOOK: backup.yml *********************************************************** Positional arguments: backup.yml verbosity: 4 connection: ssh become_method: sudo tags: ('all',) check: True inventory: ('/builds/root/aci-epg/hosts.yaml',) extra_vars: ('password=password# Input_file=epg_input.csv',) forks: 5 1 plays in backup.yml PLAY [Prompt for host] ********************************************************* TASK [Create Fabric backup] **************************************************** task path: /builds/root/aci-epg/backup.yml:18 redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root <127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0' <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041 `" && echo ansible-tmp-1743450706.4921732-299-54234027125041="` echo /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041 `" ) && sleep 0' redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest Using module file /venv/lib/python3.12/site-packages/ansible_collections/cisco/aci/plugins/modules/aci_rest.py <127.0.0.1> PUT /root/.ansible/tmp/ansible-local-296g7to8iwo/tmp5aco7tw8 TO /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041/AnsiballZ_aci_rest.py <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041/ /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041/AnsiballZ_aci_rest.py && sleep 0' <127.0.0.1> EXEC /bin/sh -c '/venv/bin/python3 /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041/AnsiballZ_aci_rest.py && sleep 0' <127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1743450706.4921732-299-54234027125041/ > /dev/null 2>&1 && sleep 0' The full traceback is: File "/tmp/ansible_aci_rest_payload_miot787v/ansible_aci_rest_payload.zip/ansible_collections/cisco/aci/plugins/module_utils/aci.py", line 520, in login self.response_json(auth["body"]) ~~~~^^^^^^^^ fatal: [localhost]: FAILED! => { "changed": false, "invocation": { "module_args": { "annotation": "orchestrator:ansible", "certificate_name": null, "content": "{\n \"configExportP\": {\n \"attributes\": {\n \"dn\": \"uni/fabric/configexp-Postman\",\n \"name\": \"Postman\",\n \"snapshot\": \"true\",\n \"targetDn\": \"\",\n \"adminSt\": \"triggered\",\n \"rn\": \"configexp-Postman\",\n \"status\": \"created,modified\",\n \"descr\": \"Fabric-Pre-change\"\n },\n \"children\":[]\n }\n}\n", "host": "IP", "method": "post", "output_level": "normal", "output_path": null, "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "path": "/api/mo/uni.json", "port": null, "private_key": null, "protocol": "https", "rsp_subtree_preserve": false, "src": null, "suppress_previous": null, "suppress_verification": null, "timeout": null, "use_proxy": null, "use_ssl": null, "username": "username", "validate_certs": false, "working_directory": "/builds/root/aci-epg" } }, "msg": "Connection failed for https://IP/api/aaaLogin.json. Request failed: <urlopen error Tunnel connection failed: 503 Service Unavailable>" } PLAY RECAP ********************************************************************* localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1 4. Sure i will check this tomorrow. From: Shreyas ***@***.***> Sent: 31 March 2025 17:38 To: CiscoDevNet/ansible-aci ***@***.***> Cc: Prasanth Velunny (JSHS R) ***@***.***>; Mention ***@***.***> Subject: Re: [CiscoDevNet/ansible-aci] aci_rest module giving 503 Service unavailable error (Issue #736) @prasanthvelunny78<https://github.com/prasanthvelunny78> Can you please post the following: 1. Command used to run the ansible backup script? 2. The complete snippet of the error message? 3. Can you re-check the proxy config used when running the Ansible backup? - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJS27DAGFM74L3QRMJT2XFVPJAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRWG44DMNBVGM>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> [shrsr]shrsr left a comment (CiscoDevNet/ansible-aci#736)<#736 (comment)> @prasanthvelunny78<https://github.com/prasanthvelunny78> Can you please post the following: 1. Command used to run the ansible backup script? 2. The complete snippet of the error message? 3. Can you re-check the proxy config used when running the Ansible backup? - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJS27DAGFM74L3QRMJT2XFVPJAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRWG44DMNBVGM>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[shrsr]

1. Shouldn't host be set to "{{ Fabric }}"?
2. Can you also please set use_proxy to True as shown below?

 - name: Create Fabric backup
   tags: BACKUP
   aci_rest:
        host: "{{ Fabric }}"
        username: "{{ username }}"
        password: "{{ password }}"
        validate_certs: False
        use_proxy: True
        path: /api/mo/uni.json
        method: post
        content: |
          {
             "configExportP": {
             "attributes": {
             "dn": "uni/fabric/configexp-Postman",
             "name": "Postman",
             "snapshot": "true",
             "targetDn": "",
             "adminSt": "triggered",
             "rn": "configexp-Postman",
             "status": "created,modified",
             "descr": "Fabric-Pre-change"
               },
             "children":[]
             }
          }

[prasanthvelunny78]

Hi Shreyas, 1. Yes, that was the original script using variables but now I added the real value instead of variable. 2. I tested with what you suggested but the same error message. Host value is picking up correctly but some problems when it reaches here File "/tmp/ansible_aci_rest_payload_288esb7g/ansible_aci_rest_payload.zip/ansible_collections/cisco/aci/plugins/module_utils/aci.py", line 520, in login self.response_json(auth["body"]) ~~~~^^^^^^^^ fatal: [localhost]: FAILED! => { Script look like this now. No variables. - name: Prompt for host hosts: localhost connection: local become: false gather_facts: false tasks: - name: Create Fabric backup tags: BACKUP aci_rest: host: IP address username: username password: password validate_certs: False use_proxy: True path: /api/mo/uni.json method: post content: | { "configExportP": { "attributes": { "dn": "uni/fabric/configexp-Postman", "name": "Postman", "snapshot": "true", "targetDn": "", "adminSt": "triggered", "rn": "configexp-Postman", "status": "created,modified", "descr": "Fabric-Pre-change" }, "children":[] } } From: Shreyas ***@***.***> Sent: 31 March 2025 21:17 To: CiscoDevNet/ansible-aci ***@***.***> Cc: Prasanth Velunny (JSHS R) ***@***.***>; Mention ***@***.***> Subject: Re: [CiscoDevNet/ansible-aci] aci_rest module giving 503 Service unavailable error (Issue #736) 1. Shouldn't host be set to "{{ Fabric }}"? 2. Can you also please set use_proxy to True as shown below? - name: Create Fabric backup tags: BACKUP aci_rest: host: "{{ Fabric }}" username: "{{ username }}" password: "{{ password }}" validate_certs: False use_proxy: True path: /api/mo/uni.json method: post content: | { "configExportP": { "attributes": { "dn": "uni/fabric/configexp-Postman", "name": "Postman", "snapshot": "true", "targetDn": "", "adminSt": "triggered", "rn": "configexp-Postman", "status": "created,modified", "descr": "Fabric-Pre-change" }, "children":[] } } - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJW7CU7N6W73BDVPQK32XGPEHAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRXGMYDAOJXGU>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> [shrsr]shrsr left a comment (CiscoDevNet/ansible-aci#736)<#736 (comment)> 1. Shouldn't host be set to "{{ Fabric }}"? 2. Can you also please set use_proxy to True as shown below? - name: Create Fabric backup tags: BACKUP aci_rest: host: "{{ Fabric }}" username: "{{ username }}" password: "{{ password }}" validate_certs: False use_proxy: True path: /api/mo/uni.json method: post content: | { "configExportP": { "attributes": { "dn": "uni/fabric/configexp-Postman", "name": "Postman", "snapshot": "true", "targetDn": "", "adminSt": "triggered", "rn": "configexp-Postman", "status": "created,modified", "descr": "Fabric-Pre-change" }, "children":[] } } - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJW7CU7N6W73BDVPQK32XGPEHAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRXGMYDAOJXGU>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[shrsr]

@prasanthvelunny78 Can you please use the namespace cisco.aci with all our modules as shown below?
You can use the following command to install the collection-> ansible-galaxy collection install cisco.aci

- name: Create Fabric backup
   tags: BACKUP
   cisco.aci.aci_rest:
        host: "{{ Fabric }}"
        username: "{{ username }}"
        password: "{{ password }}"
        validate_certs: False
        use_proxy: True
        path: /api/mo/uni.json
        method: post
        content: |
          {
             "configExportP": {
             "attributes": {
             "dn": "uni/fabric/configexp-Postman",
             "name": "Postman",
             "snapshot": "true",
             "targetDn": "",
             "adminSt": "triggered",
             "rn": "configexp-Postman",
             "status": "created,modified",
             "descr": "Fabric-Pre-change"
               },
             "children":[]
             }
          }

- name: Create EPG
   cisco.aci.aci_epg:
        host: "{{ Fabric }}"
        username: username
        password: "{{ password }}"
        tenant: "{{ item.Tenant }}"
        ap: "{{ item.AP }}"
        epg: "{{ item.EPG }}"
        bd: "{{ item.Bridge_domain }}"
        description: "{{ item.Description }}"
        state: "{{ item.State }}"
        validate_certs: no

Alternatively since you're only facing issue with cisco.aci.aci_rest, can you please try using our dedicated module?

- name: Add a Configuration Export Policy
  cisco.aci.aci_config_export_policy:
    host: apic
    username: admin
    password: SomeSecretPassword
    name: Postman
    snapshot: true
    start_now: triggered
    description: Fabric-Pre-change
    state: present
  delegate_to: localhost

[prasanthvelunny78]

Hi Shreyas, Please find attached logs. Please note that, I removed the actual value of proxy, apic_ip etc due to our internal security reasons. Many Thanks Prasanth From: Shreyas ***@***.***> Sent: 31 March 2025 21:41 To: CiscoDevNet/ansible-aci ***@***.***> Cc: Prasanth Velunny (JSHS R) ***@***.***>; Mention ***@***.***> Subject: Re: [CiscoDevNet/ansible-aci] aci_rest module giving 503 Service unavailable error (Issue #736) @prasanthvelunny78<https://github.com/prasanthvelunny78> Can you please the post the complete error message for this one? - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJVNNNXMMB4QH4QZIXT2XGR7DAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRXGM2DOOBTGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> [shrsr]shrsr left a comment (CiscoDevNet/ansible-aci#736)<#736 (comment)> @prasanthvelunny78<https://github.com/prasanthvelunny78> Can you please the post the complete error message for this one? - Reply to this email directly, view it on GitHub<#736 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BPLINJVNNNXMMB4QH4QZIXT2XGR7DAVCNFSM6AAAAAB2EHBL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRXGM2DOOBTGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> Running with gitlab-runner 17.6.1 (6826a62f) on lbg_shared_runners_secondary t1_GDCsYG, system ID: s_634694b51e33 Resolving secrets Preparing the "docker" executor 00:03 Using Docker executor with image python:3.12-slim ... Pulling docker image python:3.12-slim ... Using docker image sha256:73bcaeb9132b14bb09076fefd88ed9169c27c9bdc0a0111dd5348af5e98e3a3a for python:3.12-slim with digest ***@***.***:a866731a6b71c4a194a845d86e06568725e430ed21821d0c52e4efb385cf6c6f ... Preparing environment 00:01 Running on runner-t1gdcsyg-project-10-concurrent-0 via blp21845001... Getting source from Git repository 00:01 Fetching changes with git depth set to 20... Reinitialized existing Git repository in /builds/root/aci-epg/.git/ Checking out c673ada7 as detached HEAD (ref is test-backup)... Skipping Git submodules setup Executing "step_script" stage of the job script 01:33 Using docker image sha256:73bcaeb9132b14bb09076fefd88ed9169c27c9bdc0a0111dd5348af5e98e3a3a for python:3.12-slim with digest ***@***.***:a866731a6b71c4a194a845d86e06568725e430ed21821d0c52e4efb385cf6c6f ... $ export http_proxy="proxy_replaced_due_seucurity_reason" $ export https_proxy="proxy_replaced_due_seucurity_reason" $ export no_proxy="localhost,127.0.0.1,gitlab.internal.domain.com,10.0.0.0/8" $ python3 -m venv /venv $ source /venv/bin/activate $ pip install requests_toolbelt Collecting requests_toolbelt Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB) Collecting requests<3.0.0,>=2.0.1 (from requests_toolbelt) Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB) Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.0.1->requests_toolbelt) Downloading charset_normalizer-3.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (35 kB) Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.0.1->requests_toolbelt) Downloading idna-3.10-py3-none-any.whl.metadata (10 kB) Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.0.1->requests_toolbelt) Downloading urllib3-2.3.0-py3-none-any.whl.metadata (6.5 kB) Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.0.1->requests_toolbelt) Downloading certifi-2025.1.31-py3-none-any.whl.metadata (2.5 kB) Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB) Downloading requests-2.32.3-py3-none-any.whl (64 kB) Downloading certifi-2025.1.31-py3-none-any.whl (166 kB) Downloading charset_normalizer-3.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (145 kB) Downloading idna-3.10-py3-none-any.whl (70 kB) Downloading urllib3-2.3.0-py3-none-any.whl (128 kB) Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, requests_toolbelt Successfully installed certifi-2025.1.31 charset-normalizer-3.4.1 idna-3.10 requests-2.32.3 requests_toolbelt-1.0.0 urllib3-2.3.0 [notice] A new release of pip is available: 24.3.1 -> 25.0.1 [notice] To update, run: pip install --upgrade pip $ pip install ansible Collecting ansible Downloading ansible-11.4.0-py3-none-any.whl.metadata (8.1 kB) Collecting ansible-core~=2.18.4 (from ansible) Downloading ansible_core-2.18.4-py3-none-any.whl.metadata (7.7 kB) Collecting jinja2>=3.0.0 (from ansible-core~=2.18.4->ansible) Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB) Collecting PyYAML>=5.1 (from ansible-core~=2.18.4->ansible) Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) Collecting cryptography (from ansible-core~=2.18.4->ansible) Downloading cryptography-44.0.2-cp39-abi3-manylinux_2_34_x86_64.whl.metadata (5.7 kB) Collecting packaging (from ansible-core~=2.18.4->ansible) Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB) Collecting resolvelib<1.1.0,>=0.5.3 (from ansible-core~=2.18.4->ansible) Downloading resolvelib-1.0.1-py2.py3-none-any.whl.metadata (4.0 kB) Collecting MarkupSafe>=2.0 (from jinja2>=3.0.0->ansible-core~=2.18.4->ansible) Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) Collecting cffi>=1.12 (from cryptography->ansible-core~=2.18.4->ansible) Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB) Collecting pycparser (from cffi>=1.12->cryptography->ansible-core~=2.18.4->ansible) Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes) Downloading ansible-11.4.0-py3-none-any.whl (54.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 MB 20.0 MB/s eta 0:00:00 Downloading ansible_core-2.18.4-py3-none-any.whl (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 19.3 MB/s eta 0:00:00 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 10.0 MB/s eta 0:00:00 Downloading resolvelib-1.0.1-py2.py3-none-any.whl (17 kB) Downloading cryptography-44.0.2-cp39-abi3-manylinux_2_34_x86_64.whl (4.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 16.5 MB/s eta 0:00:00 Downloading packaging-24.2-py3-none-any.whl (65 kB) Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB) Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23 kB) Downloading pycparser-2.22-py3-none-any.whl (117 kB) Installing collected packages: resolvelib, PyYAML, pycparser, packaging, MarkupSafe, jinja2, cffi, cryptography, ansible-core, ansible Successfully installed MarkupSafe-3.0.2 PyYAML-6.0.2 ansible-11.4.0 ansible-core-2.18.4 cffi-1.17.1 cryptography-44.0.2 jinja2-3.1.6 packaging-24.2 pycparser-2.22 resolvelib-1.0.1 [notice] A new release of pip is available: 24.3.1 -> 25.0.1 [notice] To update, run: pip install --upgrade pip $ ansible --version ansible [core 2.18.4] config file = /builds/root/aci-epg/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /venv/lib/python3.12/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /venv/bin/ansible python version = 3.12.9 (main, Mar 17 2025, 23:25:08) [GCC 12.2.0] (/venv/bin/python3) jinja version = 3.1.6 libyaml = True $ ansible-galaxy collection install cisco.aci Starting galaxy collection install process Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`. $ ansible-galaxy collection list # /venv/lib/python3.12/site-packages/ansible_collections Collection Version ---------------------------------------- ------- amazon.aws 9.3.0 ansible.netcommon 7.1.0 ansible.posix 1.6.2 ansible.utils 5.1.2 ansible.windows 2.8.0 arista.eos 10.1.1 awx.awx 24.6.1 azure.azcollection 3.3.1 check_point.mgmt 6.4.0 chocolatey.chocolatey 1.5.3 cisco.aci 2.10.1 cisco.asa 6.1.0 cisco.dnac 6.31.0 cisco.intersight 2.0.20 cisco.ios 9.1.2 cisco.iosxr 10.3.0 cisco.ise 2.10.0 cisco.meraki 2.20.8 cisco.mso 2.9.0 cisco.nxos 9.3.0 cisco.ucs 1.15.0 cloud.common 4.0.0 cloudscale_ch.cloud 2.4.1 community.aws 9.1.0 community.ciscosmb 1.0.10 community.crypto 2.26.0 community.digitalocean 1.27.0 community.dns 3.2.2 community.docker 4.5.2 community.general 10.5.0 community.grafana 2.1.0 community.hashi_vault 6.2.0 community.hrobot 2.2.0 community.library_inventory_filtering_v1 1.0.2 community.libvirt 1.3.1 community.mongodb 1.7.9 community.mysql 3.13.0 community.network 5.1.0 community.okd 4.0.1 community.postgresql 3.12.0 community.proxysql 1.6.0 community.rabbitmq 1.4.0 community.routeros 3.5.0 community.sap_libs 1.4.2 community.sops 2.0.3 community.vmware 5.5.0 community.windows 2.4.0 community.zabbix 3.3.0 containers.podman 1.16.3 cyberark.conjur 1.3.3 cyberark.pas 1.0.30 dellemc.enterprise_sonic 2.5.1 dellemc.openmanage 9.10.0 dellemc.powerflex 2.6.0 dellemc.unity 2.0.0 f5networks.f5_modules 1.34.1 fortinet.fortimanager 2.9.1 fortinet.fortios 2.3.9 google.cloud 1.5.1 grafana.grafana 5.7.0 hetzner.hcloud 4.3.0 ibm.qradar 4.0.0 ibm.spectrum_virtualize 2.0.0 ibm.storage_virtualize 2.6.0 ieisystem.inmanage 3.0.0 infinidat.infinibox 1.4.5 infoblox.nios_modules 1.8.0 inspur.ispim 2.2.3 junipernetworks.junos 9.1.0 kaytus.ksmanage 2.0.0 kubernetes.core 5.1.0 kubevirt.core 2.1.0 lowlydba.sqlserver 2.5.0 microsoft.ad 1.8.0 netapp.cloudmanager 21.24.0 netapp.ontap 22.14.0 netapp.storagegrid 21.14.0 netapp_eseries.santricity 1.4.1 netbox.netbox 3.21.0 ngine_io.cloudstack 2.5.0 openstack.cloud 2.4.1 ovirt.ovirt 3.2.0 purestorage.flasharray 1.33.1 purestorage.flashblade 1.19.2 sensu.sensu_go 1.14.0 splunk.es 4.0.0 telekom_mms.icinga_director 2.2.2 theforeman.foreman 4.2.0 vmware.vmware 1.10.1 vmware.vmware_rest 4.6.0 vultr.cloud 1.13.0 vyos.vyos 5.0.0 wti.remote 1.0.10 $ apt-get update Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB] Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB] Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB] Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8792 kB] Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [512 B] Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [250 kB] Fetched 9297 kB in 2s (4088 kB/s) Reading package lists... $ apt-get install -y curl Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: libbrotli1 libcurl4 libldap-2.5-0 libldap-common libnghttp2-14 libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 publicsuffix Suggested packages: libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql The following NEW packages will be installed: curl libbrotli1 libcurl4 libldap-2.5-0 libldap-common libnghttp2-14 libpsl5 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 publicsuffix 0 upgraded, 13 newly installed, 0 to remove and 1 not upgraded. Need to get 1839 kB of archives. After this operation, 4550 kB of additional disk space will be used. Get:1 http://deb.debian.org/debian bookworm/main amd64 libbrotli1 amd64 1.0.9-2+b6 [275 kB] Get:2 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg-10 [20.3 kB] Get:3 http://deb.debian.org/debian bookworm/main amd64 libsasl2-2 amd64 2.1.28+dfsg-10 [59.7 kB] Get:4 http://deb.debian.org/debian bookworm/main amd64 libldap-2.5-0 amd64 2.5.13+dfsg-5 [183 kB] Get:5 http://deb.debian.org/debian bookworm/main amd64 libnghttp2-14 amd64 1.52.0-1+deb12u2 [73.0 kB] Get:6 http://deb.debian.org/debian bookworm/main amd64 libpsl5 amd64 0.21.2-1 [58.7 kB] Get:7 http://deb.debian.org/debian bookworm/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2+b2 [60.8 kB] Get:8 http://deb.debian.org/debian bookworm/main amd64 libssh2-1 amd64 1.10.0-3+b1 [179 kB] Get:9 http://deb.debian.org/debian bookworm/main amd64 libcurl4 amd64 7.88.1-10+deb12u12 [391 kB] Get:10 http://deb.debian.org/debian bookworm/main amd64 curl amd64 7.88.1-10+deb12u12 [315 kB] Get:11 http://deb.debian.org/debian bookworm/main amd64 libldap-common all 2.5.13+dfsg-5 [29.3 kB] Get:12 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules amd64 2.1.28+dfsg-10 [66.6 kB] Get:13 http://deb.debian.org/debian bookworm/main amd64 publicsuffix all 20230209.2326-1 [126 kB] debconf: delaying package configuration, since apt-utils is not installed Fetched 1839 kB in 0s (5274 kB/s) Selecting previously unselected package libbrotli1:amd64. (Reading database ... 6686 files and directories currently installed.) Preparing to unpack .../00-libbrotli1_1.0.9-2+b6_amd64.deb ... Unpacking libbrotli1:amd64 (1.0.9-2+b6) ... Selecting previously unselected package libsasl2-modules-db:amd64. Preparing to unpack .../01-libsasl2-modules-db_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package libsasl2-2:amd64. Preparing to unpack .../02-libsasl2-2_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-2:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package libldap-2.5-0:amd64. Preparing to unpack .../03-libldap-2.5-0_2.5.13+dfsg-5_amd64.deb ... Unpacking libldap-2.5-0:amd64 (2.5.13+dfsg-5) ... Selecting previously unselected package libnghttp2-14:amd64. Preparing to unpack .../04-libnghttp2-14_1.52.0-1+deb12u2_amd64.deb ... Unpacking libnghttp2-14:amd64 (1.52.0-1+deb12u2) ... Selecting previously unselected package libpsl5:amd64. Preparing to unpack .../05-libpsl5_0.21.2-1_amd64.deb ... Unpacking libpsl5:amd64 (0.21.2-1) ... Selecting previously unselected package librtmp1:amd64. Preparing to unpack .../06-librtmp1_2.4+20151223.gitfa8646d.1-2+b2_amd64.deb ... Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... Selecting previously unselected package libssh2-1:amd64. Preparing to unpack .../07-libssh2-1_1.10.0-3+b1_amd64.deb ... Unpacking libssh2-1:amd64 (1.10.0-3+b1) ... Selecting previously unselected package libcurl4:amd64. Preparing to unpack .../08-libcurl4_7.88.1-10+deb12u12_amd64.deb ... Unpacking libcurl4:amd64 (7.88.1-10+deb12u12) ... Selecting previously unselected package curl. Preparing to unpack .../09-curl_7.88.1-10+deb12u12_amd64.deb ... Unpacking curl (7.88.1-10+deb12u12) ... Selecting previously unselected package libldap-common. Preparing to unpack .../10-libldap-common_2.5.13+dfsg-5_all.deb ... Unpacking libldap-common (2.5.13+dfsg-5) ... Selecting previously unselected package libsasl2-modules:amd64. Preparing to unpack .../11-libsasl2-modules_2.1.28+dfsg-10_amd64.deb ... Unpacking libsasl2-modules:amd64 (2.1.28+dfsg-10) ... Selecting previously unselected package publicsuffix. Preparing to unpack .../12-publicsuffix_20230209.2326-1_all.deb ... Unpacking publicsuffix (20230209.2326-1) ... Setting up libpsl5:amd64 (0.21.2-1) ... Setting up libbrotli1:amd64 (1.0.9-2+b6) ... Setting up libsasl2-modules:amd64 (2.1.28+dfsg-10) ... Setting up libnghttp2-14:amd64 (1.52.0-1+deb12u2) ... Setting up libldap-common (2.5.13+dfsg-5) ... Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ... Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... Setting up libsasl2-2:amd64 (2.1.28+dfsg-10) ... Setting up libssh2-1:amd64 (1.10.0-3+b1) ... Setting up publicsuffix (20230209.2326-1) ... Setting up libldap-2.5-0:amd64 (2.5.13+dfsg-5) ... Setting up libcurl4:amd64 (7.88.1-10+deb12u12) ... Setting up curl (7.88.1-10+deb12u12) ... Processing triggers for libc-bin (2.36-9+deb12u10) ... $ curl -kI https://APIC_IP/api/aaaLogin.json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 HTTP/1.1 200 OK Server: Cisco APIC Date: Tue, 01 Apr 2025 08:51:33 GMT Content-Type: text/html; charset=utf-8 Content-Length: 0 Connection: keep-alive Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, DevCookie, APIC-challenge, Request-Tag Access-Control-Allow-Methods: POST,GET,OPTIONS,DELETE X-Frame-Options: SAMEORIGIN Strict-Transport-Security: max-age=31536000; includeSubDomains Cache-Control: no-cache Client-Cert-Enabled: false X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Content-Security-Policy: default-src 'self'; script-src 'unsafe-inline' 'unsafe-eval' 'self'; style-src blob: 'unsafe-inline' 'self'; object-src 'none'; base-uri 'self'; connect-src 'self'; font-src 'self' data:; frame-src 'self' players.brightcove.net; img-src 'self' data:; manifest-src 'self'; media-src 'self'; worker-src blob: 'self'; Access-Control-Allow-Credentials: false Expires: Thu, 01 Jan 1970 00:00:01 GMT $ ansible-playbook -i hosts.yaml backup.yml -vvvv --check --extra-vars "password=$password Input_file=epg_input.csv" ansible-playbook [core 2.18.4] config file = /builds/root/aci-epg/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /venv/lib/python3.12/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /venv/bin/ansible-playbook python version = 3.12.9 (main, Mar 17 2025, 23:25:08) [GCC 12.2.0] (/venv/bin/python3) jinja version = 3.1.6 libyaml = True Using /builds/root/aci-epg/ansible.cfg as config file setting up inventory plugins Loading collection ansible.builtin from host_list declined parsing /builds/root/aci-epg/hosts.yaml as it did not pass its verify_file() method script declined parsing /builds/root/aci-epg/hosts.yaml as it did not pass its verify_file() method Parsed /builds/root/aci-epg/hosts.yaml inventory source with yaml plugin redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest Loading collection cisco.aci from /venv/lib/python3.12/site-packages/ansible_collections/cisco/aci Loading callback plugin default of type stdout, v2.0 from /venv/lib/python3.12/site-packages/ansible/plugins/callback/default.py Skipping callback 'default', as we already have a stdout callback. Skipping callback 'minimal', as we already have a stdout callback. Skipping callback 'oneline', as we already have a stdout callback. PLAYBOOK: backup.yml *********************************************************** Positional arguments: backup.yml verbosity: 4 connection: ssh become_method: sudo tags: ('all',) check: True inventory: ('/builds/root/aci-epg/hosts.yaml',) extra_vars: ('password=pwd Input_file=epg_input.csv',) forks: 5 1 plays in backup.yml PLAY [Prompt for host] ********************************************************* TASK [Create Fabric backup] **************************************************** task path: /builds/root/aci-epg/backup.yml:18 redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root <127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0' <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447 `" && echo ansible-tmp-1743497493.6815307-299-220876132330447="` echo /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447 `" ) && sleep 0' redirecting (type: modules) ansible.builtin.aci_rest to cisco.aci.aci_rest Using module file /venv/lib/python3.12/site-packages/ansible_collections/cisco/aci/plugins/modules/aci_rest.py <127.0.0.1> PUT /root/.ansible/tmp/ansible-local-296oy4zu0ul/tmposx51f5x TO /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447/AnsiballZ_aci_rest.py <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447/ /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447/AnsiballZ_aci_rest.py && sleep 0' <127.0.0.1> EXEC /bin/sh -c '/venv/bin/python3 /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447/AnsiballZ_aci_rest.py && sleep 0' <127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1743497493.6815307-299-220876132330447/ > /dev/null 2>&1 && sleep 0' The full traceback is: File "/tmp/ansible_aci_rest_payload_nv2hmqb_/ansible_aci_rest_payload.zip/ansible_collections/cisco/aci/plugins/module_utils/aci.py", line 520, in login self.response_json(auth["body"]) ~~~~^^^^^^^^ fatal: [localhost]: FAILED! => { "changed": false, "invocation": { "module_args": { "annotation": "orchestrator:ansible", "certificate_name": null, "content": "{\n \"configExportP\": {\n \"attributes\": {\n \"dn\": \"uni/fabric/configexp-Postman\",\n \"name\": \"Postman\",\n \"snapshot\": \"true\",\n \"targetDn\": \"\",\n \"adminSt\": \"triggered\",\n \"rn\": \"configexp-Postman\",\n \"status\": \"created,modified\",\n \"descr\": \"Fabric-Pre-change-test\"\n },\n \"children\":[]\n }\n}\n", "host": "APIC_IP", "method": "post", "output_level": "normal", "output_path": null, "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "path": "/api/mo/uni.json", "port": null, "private_key": null, "protocol": "https", "rsp_subtree_preserve": false, "src": null, "suppress_previous": null, "suppress_verification": null, "timeout": null, "use_proxy": true, "use_ssl": null, "username": "USERNMAE", "validate_certs": false, "working_directory": "/builds/root/aci-epg" } }, "msg": "Connection failed for https://APIC_IP/api/aaaLogin.json. Request failed: <urlopen error timed out>" } PLAY RECAP ********************************************************************* localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1

[shrsr]

@prasanthvelunny78
Can you use this task for backup against the same inventory instead of aci_rest?

- name: Add a Configuration Export Policy
  cisco.aci.aci_config_export_policy:
    host: apic
    username: admin
    password: SomeSecretPassword
    name: Postman
    snapshot: true
    start_now: triggered
    description: Fabric-Pre-change
    validate_certs: false
    state: present

[shrsr]

@prasanthvelunny78 Another question: Is your ansible.cfg in the same location as the ansible backup script?

ANSIBLE_CONFIG: 
ansible.cfg
https_proxy: "http://myproxy.com/"
no_proxy: "localhost,10.0.0.0/8"

[shrsr]

@prasanthvelunny78 Were you able to resolve this with ansible.cfg?

[lhercot]

@prasanthvelunny78 any updates on @shrsr proposal?