How to troubleshoot HTTP 400 response codes from ingress-nginx ingresses

This document (000020074) is provided subject to the disclaimer at the end of this document.

Situation

Task

This article details how to troubleshoot failing requests, with a HTTP 400 response code, when using an ingress to access a service in a Kubernetes cluster.

This error message is typically returned due to a bad request, or as a result of an issue with the request headers or cookies.

This article is not intended to be exhaustive as there are a wide variety of causes, however some possible issues are covered.

Pre-requisites
  • A Rancher Kubernetes Engine (RKE) CLI or Rancher v2.x provisioned Kubernetes cluster
Resolution
Large request headers exceeding header buffer

Nginx has a client header buffer configuration of 4x 8KB by default. Per the Nginx documentation:

  • "A request line cannot exceed the size of one buffer, or the 414 (Request-URI Too Large) error is returned to the client."
  • "A request header field cannot exceed the size of one buffer as well, or the 400 (Bad Request) error is returned to the client."

The buffer sizes can be extended by defining the large-client-header-buffers option in the nginx-configuration ConfigMap (see below).

Large URI is duplicated into a new header passed to the backend

This issue is commonly seen where the app itself responds when queried directly with a large path, but returns a 400 error when queried through an ingress.

By default, when ingress-nginx receives a request, it adds the original request's URI to the X-Original-Uri header that it passes on to the backend. This can result in the app being unable to handle the large sized headers, in addition to the long path.

This behaviour can be disabled by setting the proxy-add-original-uri-header option to false in your nginx-configuration ConfigMap (see below).

Adding options to the ingress-nginx nginx-configuration ConfigMap

This configuration map is populated by RKE from configuration defined in the cluster config:

If the cluster is provisioned by Rancher v2.x:
  1. Edit the cluster (navigate to the cluster within Rancher, select the triple-dot button and then "Edit")
  2. Select "Edit as YAML" to open the cluster configuration as YAML, instead of a form.
  3. Add the desired configuration within the ingress block in the following format:

      ingress:
        provider: nginx
        options:
          name: value

    Rancher config

  4. Save the cluster configuration changes. RKE will go through and apply the config defined during its update process.

If the cluster is provisioned by the RKE CLI:

The process is largely the same as the Rancher process above, but the configuration is defined in the cluster.yml for this cluster:

  1. Open the cluster configuration yaml with your editor and add the ingress.options block:

    yaml ingress: provider: nginx options: name: value

    RKE config

  2. Apply this config with rke up --config <cluster configuration yaml>

Ensure you have an up-to-date cluster.rkestate within the same directory before running rke up

Further reading

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000020074
  • Creation Date: 06-May-2021
  • Modified Date:06-May-2021
    • SUSE Rancher

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center