SUSE Support

Here When You Need Us

Incomplete Removal of Proxy Setting Variable by Rancher on Rancher provisioned downstream cluster.

This document (000021825) is provided subject to the disclaimer at the end of this document.

Environment

Rancher version: 2.9.x or 2.10.x


Situation

Following the decommissioning of a proxy server and the subsequent removal of its configurations from the Rancher UI via the "Cluster Config > Agent Environment Variables" for Rancher provisioned downstream clusters, a failure to propagate these changes to the downstream nodes will result in image pull failures.

These failures may manifest as ImagePullBackOff errors or specific authentication errors such as "Error response from daemon: Get "https://registry.xyz.com/v2/": Forbidden" since the request to the Private Registry happens via the Proxy server which does not exist anymore. 

Resolution

Upon removing Proxy settings from the Rancher UI for downstream clusters via "Cluster Config > Agent Environment Variables", verification of proxy setting removal on each downstream node is required if the image pull still continues to use Proxy configurations to establish connection to Private Registry. Inspect the /var/lib/rancher/rke2/system-agent-installer/rke2-sa.env file for any remaining proxy configurations. If present, this signifies a failure of the rancher-system-agent to update its configuration from the upstream Rancher instance.

In this scenario, examine /etc/systemd/system/rancher-system-agent.env and /etc/systemd/system/rancher-system-agent.service for proxy settings. If found, these settings must be removed, and the rancher-system-agent service, along with the rke2-server or rke2-agent service, should be restarted to enforce the updated configuration.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000021825
  • Creation Date: 06-May-2025
  • Modified Date:22-May-2025
    • SUSE Rancher

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com

tick icon

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

tick icon

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.

tick icon

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.