SUSE Cloud Application Platform 2.1.1 Release Notes
This document provides guidance and an overview to high-level general features and updates for SUSE Cloud Application Platform 2.1.1. It also describes capabilities and limitations of SUSE Cloud Application Platform 2.1.1. For detailed information about deploying this product, see the Deployment Guide at https://documentation.suse.com/suse-cap/2.0/single-html/cap-guides/#part-cap-deployment.
These release notes are updated periodically. The latest version of these release notes is always available at https://www.suse.com/releasenotes. General documentation can be found at https://documentation.suse.com/suse-cap/2.
1 About SUSE Cloud Application Platform #
SUSE Cloud Application Platform is a modern application delivery platform used to bring an advanced cloud native developer experience to Kubernetes—the de-facto standard for enterprise container orchestration. SUSE Cloud Application Platform eliminates manual IT configuration and helps accelerate innovation by getting applications to market faster. Developers can serve themselves and get apps to the cloud in minutes instead of weeks, while staying within IT guidelines, and without relying on scarce IT resources to perform manual configuration each step of the way. Streamlining application delivery opens a clear path to increased business agility, led by enterprise development, operations, and DevOps teams.
SUSE Cloud Application Platform increases business agility by helping enterprises to:
Boost developer productivity with easy one step deployment of cloud native applications using the language and framework most appropriate for the task.
Reduce complexity and increase IT efficiency with a single, lean, platform that brings together proven open source technologies for rapid application delivery at scale.
Maximize return on investment with industry leading open-source technologies that leverage your existing investments.
2 Support Statement for SUSE Cloud Application Platform #
To receive support, you need an appropriate subscription with SUSE. For more information, see https://www.suse.com/support/?id=SUSE_Cloud_Application_Platform.
The following definitions apply:
2.1 Version Support #
Technical Support and Troubleshooting (L1 - L2): SUSE will provide technical support and troubleshooting for version 2.1 until May 31, 2022.
Patches and updates (L3): SUSE will provide patches and updates for 2.1 (e.g. 2.1.1, 2.1.2) to resolve critical bugs or address high severity security issues. The patches may include updates from upstream Cloud Foundry releases.
SUSE Cloud Application Platform closely follows upstream Cloud Foundry releases which may implement fixes and changes which are not backwards compatible with previous releases. SUSE will backport patches for critical bugs and security issues on a best efforts basis.
2.2 Platform Support #
SUSE Cloud Application Platform is fully supported on Amazon EKS, Microsoft Azure AKS, Google GKE, Rancher Kubernetes Engine (RKE), and RKE Government (RKE2). Each release is tested by SUSE Cloud Application Platform QA on these platforms.
SUSE Cloud Application Platform is fully supported on SUSE CaaS Platform, wherever it happens to be installed. If SUSE CaaS Platform is supported on a particular CSP, the customer can get support for SUSE Cloud Application Platform in that context.
SUSE can provide support for SUSE Cloud Application Platform on 3rd party/generic Kubernetes on a case-by-case basis provided:
the Kubernetes cluster satisfies the Requirements listed here: https://documentation.suse.com/suse-cap/2.0/html/cap-guides/cha-cap-depl-kube-requirements.html#sec-cap-changes-kube-reqs
The
kube-ready-state-check.sh
script has been run on the target Kubernetes cluster and does not show any configuration problemsa SUSE Services or Sales Engineer has verified that SUSE Cloud Application Platform works correctly on the target Kubernetes cluster
Any incident with SUSE Cloud Application Platform is also fully supported as long as the problem can be replicated on SUSE CaaS Platform, AKS, Amazon EKS or GKE. Bugs identified on 3rd party / generic Kubernetes which are unique to that platform and can not be replicated on the core supported platforms are fixed on a best efforts basis. SUSE will not replicate the deployed Kubernetes environment internally in order to reproduce errors.
SUSE will only support the usage of original packages. That is, packages that are unchanged and not recompiled.
3 Major Changes #
3.1 Release 2.1.1, March 2021 #
3.1.1 What Is New? #
KubeCF has been updated to version 2.7.13:
Several components and dependencies were updated to address bugs and CVEs
For a full list of features and fixes, see the links in Section 3.1.2, “Component Versions”.
cf-operator has been updated to version 7.2.1:
The upstream project was renamed to quarks-operator. The name change does not affect SUSE Cloud Application Platform, where the component will still be referred to as the cf-operator
Fixes included for multi-AZ
Bumped for use with Golang 1.15.8
For a full list of features and fixes, see https://github.com/cloudfoundry-incubator/quarks-operator/releases/tag/v7.2.1
Stratos Console has been updated to version 4.4.1:
Added support to deploy to clusters with Pod Security Policies enabled
For a full list of features and fixes, see the links in Section 3.1.2, “Component Versions”.
Minibroker has been updated to version 1.2.0:
Default Helm chart repository updated to https://charts.helm.sh/stable
For a full list of features and fixes, see https://github.com/kubernetes-sigs/minibroker/releases/tag/v1.2.0
3.1.2 Component Versions #
cf-operator: 7.2.1+0.gaeb6ef3
Updated from 6.1.17+0.gec409fd7. The list of releases since then:
KubeCF: 2.7.13
Updated from 2.5.8. The list of releases since then:
Stratos Console: 4.4.1
Updated from 4.2.0. The list of releases since then:
Stratos Metrics: 1.3.0
Minibroker: 1.2.0
3.1.3 Features and Fixes #
Bumped cf-operator to version 7.2.1 which contains several bug fixes
Bumped stemcell version for SUSE buildpacks to 29.6
Bumped
suse-java-buildpack
release to 4.36.0Bumped
sle15
stack release to 26.14Fixed issue where restarting a pod with multiple instances would fail due to an incorrect label value
Includes these Cloud Foundry component versions:
app-autoscaler: 3.0.1
bits-services: 2.28.0
capi: 1.98.0
cf-acceptance-tests: 0.0.22
cf-deployment: 13.17
cf-smoke-tests: 41.0.1
cf-syslog-drain: 10.2.11
cflinuxfs3: 0.203.0
credhub: 2.8.0
diego: 2.48.0
eirini: 1.8.0
garden-runc: 1.19.16
loggregator: 106.3.10
loggregator-agent: 6.1.1
log-cache: 2.8.0
nats: 39
postgres: 39
routing: 0.206.0
scf-helper: 1.0.13
silk: 2.33.0
sle15: 26.14
statsd-injector: 1.11.15
sync-integration-tests: 0.0.3
uaa: 74.24.0
Buildpacks:
binary-buildpack: 1.0.36
dotnetcore-buildpack: 2.3.18
go-buildpack: 1.9.23
java-buildpack: 4.36.0
nginx-buildpack: 1.1.18
nodejs-buildpack: 1.7.35
php-buildpack: 4.4.26
python-buildpack: 1.7.26
staticfile-buildpack: 1.5.13
ruby-buildpack: 1.8.27
3.1.4 Known Issues #
3.1.5 Deprecations #
This will be the final release with Eirini as a supported scheduler. As of the next minor release of KubeCF, we will be deprecating Eirini with Diego planned as the only supported scheduler. If you currently run Eirini as a scheduler and plan to upgrade in the future, please plan for this change. Eirini will still be included as is, but if problems arise when bumping cf-deployment versions to address CVEs, it may be removed without further notice.
This will be the final release with
sle15
as a supported stack. Going forward,cflinuxfs3
will be the only supported stack and we will no longer buld updated rootfs and buildpacks withsle15
. If you plan to upgrade in the future, please start planning to migrate applications to rely on thecflinuxfs3
stack instead.This will be the final release containing updates to the bundled buildpacks.
3.2 Release 2.1.0, October 2020 #
3.2.1 What Is New? #
KubeCF has been updated to version 2.5.8:
Eirini has graduated from technical preview
Introduced multi-stack support
Enabled c2c networking
Expanded information on the more commonly used entries in values.yaml
Bumped cf-deployment to 13.17
Allow tolerations to be set for the instance group pods
Ability to set memory limits and requests with defaults now set
Added labels to QuarksStatefulSets
For a full list of features and fixes, see the links in Section 3.2.2, “Component Versions”.
cf-operator has been updated to version 6.1.17:
Improved memory limit validation
Added support for CPU limits
Added log rotation support in the log container
For a full list of features and fixes, see https://github.com/cloudfoundry-incubator/quarks-operator/releases/tag/v6.1.17
Stratos Console has been updated to version 4.2.0:
Configurable NGINX protocols and ciphers using Helm values
ArtifactHub replaces Helm Hub
Added support for API keys
For a full list of features and fixes, see the links in Section 3.2.2, “Component Versions”.
Stratos Console Metrics has been updated to version 1.3.0:
Bumped Prometheus version to 2.20.1 and Helm Chart Version 11.15.0
Bumped CF Firehose exporter to version 6.1.0
For a full list of features and fixes, see https://github.com/SUSE/stratos-metrics/releases/tag/1.3.0
Minibroker has been updated to version 1.1.0:
Relies on Helm 3 internally
Added support for RabbitMQ
Added support for asynchronous operations
Admins can set override parameters that will prevent users from setting any parameters during provisioning
For a full list of features and fixes, see https://github.com/kubernetes-sigs/minibroker/releases/tag/v1.1.0
3.2.2 Component Versions #
cf-operator: 6.1.17+0.gec409fd7
KubeCF: 2.5.8
Updated from 2.2.3. The list of releases since then:
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.8
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.7
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.6
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.5
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.4
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.3
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.5.0
https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.4.0
Stratos Console: 4.2.0
Updated from 4.0.1. The list of releases since then:
Stratos Metrics: 1.3.0
Minibroker: 1.1.0
3.2.3 Features and Fixes #
For cf-operator,
--set "global.singleNamespace.name=kubecf"
replaces the previous--set "global.operator.watchNamespace=kubecf"
Improved startup dependency declarations
Diego cells will always use a hostpath storage class
Include Eirini, EiriniX and Bits templates in KubeCF
UAA will include the cf-cli job only if Credhub is enabled
Bumped PXC to 5.7.30-33
Includes these Cloud Foundry component versions:
app-autoscaler: 3.0.1
bits-services: 2.28.0
capi: 1.98.0
cf-acceptance-tests: 0.0.22
cf-deployment: 13.17
cf-smoke-tests: 41.0.1
cf-syslog-drain: 10.2.11
cflinuxfs3: 0.203.0
credhub: 2.8.0
diego: 2.48.0
eirini: 1.8.0
garden-runc: 1.19.16
loggregator: 106.3.10
loggregator-agent: 6.1.1
log-cache: 2.8.0
nats: 34
postgres: 39
routing: 0.206.0
scf-helper: 1.0.13
silk: 2.33.0
sle15: 10.93
statsd-injector: 1.11.15
sync-integration-tests: 0.0.3
uaa: 74.24.0
Buildpacks:
binary-buildpack: 1.0.36
dotnetcore-buildpack: 2.3.16
go-buildpack: 1.9.19
java-buildpack: 4.32.1
nginx-buildpack: 1.1.15
nodejs-buildpack: 1.7.30
php-buildpack: 4.4.22
python-buildpack: 1.7.23
staticfile-buildpack: 1.5.12
ruby-buildpack: 1.8.25
3.2.4 Known Issues #
During the upgrades to 2.1, there will be some downtime for apps.
If you are using an HA setup of the internal database in CAP 2.0.x (or KubeCF), you will need to scale down
sizing.database.instances
to 1 in order to upgrade to CAP 2.1. Running a high available version of the internal database during the upgrade will result in confusion during the password rotation process and you will run into difficulties recovering from it.If you are planning to convert from Diego to Eirini, please upgrade your Diego environment first to CAP 2.1 and then migrate to Eirini as the earlier CAP versions relied on a technical preview version of Eirini.
If you are running CAP 2.0.x with Diego with apps relying on the
cflinuxfs3
stack and plan on migrating to Eirini with CAP 2.1, you will need to convert your apps to use thesle15
stack. You can re-push your apps withcf push -s sle15
if thecflinuxfs3
stack was used; otherwise your apps will crash on Eirini.Eirini apps will require slightly more memory than their Diego equivalent; from what has been tested, add an additional 32MB into the Eirini manifest.
TCP routing is not available in Eirini deployments at this time.
When converting from Diego to Eirini, you will see older Diego pods up for several minutes in the midst of the migration until the operator cleans up the older deployment. You will still expect to see a
diego-api
pod since that’s where locket runs.Eirini requires the
k8s-metrics-server
to be installed on the Kubernetes environment where CAP is installed in order for Stratos Metrics to work.Stratos Metrics will not show disk stats on Eirini.
When there is a Kubernetes outage, Eirini will not properly automatically restart apps upon its return. You will need to manually start them up at present.
log-cache
will need to have setmemory_limit_percent: 3
as a workaround to allocate enough memory for that to run within a safe limit without interfering with apps. See https://documentation.suse.com/suse-cap/2.1/single-html/cap-guides/#sec-cap-tbl-log-cache-memoryThe v3 API used by cf-cli v7 should not be interchangeable with the v2 API (cf-cli v6) based on certain functionality that is not compatible. Stratos Console also has problems relying on v3 API and in comparison to v2, performance degradation is expected
Support for public cloud service brokers was removed as most of those OSBAPI-based brokers have been deprecated in lieu of the various public clouds' own in-house solutions.
3.3 Release 2.0.1, August 2020 #
3.3.1 What Is New? #
KubeCF has been updated to version 2.2.3:
For a full list of features and fixes see https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.2.3
cf-operator has been updated to version 4.5.13:
For a full list of features and fixes see https://github.com/cloudfoundry-incubator/quarks-operator/releases/tag/v4.5.13
Stratos Console has been updated to version 4.0.1:
Configurable NGINX protocols and ciphers using Helm values
For a full list of features and fixes see https://github.com/SUSE/stratos/blob/master/CHANGELOG.md#401.
3.3.2 Features and Fixes #
Fixed issue where logs in the
cloud_controller
container in theapi
pod were not being rotatingDefault podAntiAffinity rules added
Ability to customize affinity/anti-affinity settings (see https://documentation.suse.com/suse-cap/2.0.1/single-html/cap-guides/#sec-cap-aks-affinity for instructions)
Includes these Cloud Foundry component versions:
app-autoscaler: 3.0.0
bits-services: 2.28.0
bosh-dns-aliases: 0.0.3
bpm: 1.1.7
capi: 1.91.0
cf-acceptance-tests: 0.0.13
cf-deployment: 12.36
cf-smoke-tests: 40.0.128
cf-syslog-drain: 10.2.11
cflinuxfs3: 0.167.0
credhub: 2.5.11
diego: 2.44.0
eirini: 0.0.27
garden-runc: 1.19.10
loggregator: 106.3.8
loggregator-agent: 5.3.7
log-cache: 2.6.8
nats: 33
postgres-release: 39
routing: 0.198.0
scf-helper: 1.0.13
silk: 2.28.0
sle15: 10.93
statsd-injector: 1.11.15
sync-integration-tests: 0.0.3
uaa: 74.15.0
Buildpacks:
binary-buildpack: 1.0.36
dotnetcore-buildpack: 2.3.9
go-buildpack: 1.9.11
java-buildpack: 4.29.1
nginx-buildpack: 1.1.7
nodejs-buildpack: 1.7.17
php-buildpack: 4.4.12
python-buildpack: 1.7.12
staticfile-buildpack: 1.5.5
ruby-buildpack: 1.8.15
3.3.3 Known Issues #
During upgrades from SUSE Cloud Application Platform 2.0 to 2.0.1, there will be a few minutes of app downtime.
SUSE Cloud Application Platform 2.0.1 works with these external databases at present: :mysql: 5.7, 5.8 and :mariadb: 10.1.x.
Only non-encrypted connections to external databases are supported. It is recommended that connections to external databases are allowed only within trusted networks. For instructions on how to connect to external databases, see https://documentation.suse.com/suse-cap/2.0.1/single-html/cap-guides/#sec-cap-caasp-external-database.
Important: Mitigating Gorouter DoS Attacks (CVE-2020-15586)
The current release of SUSE Cloud Application Platform is affected by CVE-2020-15586 whereby the Gorouter is vulnerable to a Denial-of-Service (DoS) attack via requests with the "Expect: 100-continue" header. For details regarding this vulnerability, see https://www.cloudfoundry.org/blog/cve-2020-15586/.
If available, operators are advised to upgrade to a SUSE Cloud Application Platform release that is not affected by this vulnerability. Always review the release notes (https://suse.com/releasenotes/) to verify whether a given SUSE Cloud Application Platform release is affected. If it is not possible to upgrade immediately, we recommend operators follow the mitigations from Cloud Foundry’s security update (see https://www.cloudfoundry.org/blog/cve-2020-15586/):
Configure an HTTP load balancer in front of the Gorouters to drop the
Expect 100-continue
header completely.This may cause delays in HTTP clients that utilize the Expect: 100 continue behavior. However, this should not affect the correctness of HTTP applications.
Configure an HTTP load balancer in front of the Gorouters to drop the
Expect: 100-continue
header and immediately respond with “100 Continue”.This may cause HTTP clients to send the request body unnecessarily in some cases where the server would have responded with a final status code before requesting the body. However, this should not affect the correctness of HTTP applications.
If you are using a TCP / L4 load balancer for your Gorouters instead of an HTTP load balancer, consider the following:
Add firewall rules to prevent traffic from any source making requests that are causing this panic.
You may use the extra_headers_to_log property to enable logging of the “Expect” request header to help identify sources of this malicious traffic.
3.4 Release 2.0, June 2020 #
3.4.1 What Is New? #
SUSE Cloud Application Platform 2 is based on KubeCF, which uses the Quarks operator, also referred to as cf-operator, to deploy releases from cf-deployment into Kubernetes environments. With this release, KubeCF replaces SCF as the Cloud Foundry Application Runtime (CFAR) component of SUSE Cloud Application Platform. Each KubeCF release must be paired with a corresponding release of the Quarks operator, a Kubernetes operator (https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) implementation. In the case of SUSE Cloud Application Platform 2.0, KubeCF 2.2.2 must be paired with cf-operator 4.5.6.
KubeCF 2.2.2 replaces {scf}:
cf-deployment has been updated to 12.36
Support for external Blobstore configuration
KubeCF 2.2.2 is paired with cf-operator 4.5.6
For a full list of features and fixes see https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.2.2
cf-operator 4.5.6 has been added:
For a full list of features and fixes see https://github.com/cloudfoundry-incubator/quarks-operator/releases/tag/v4.5.6
Stratos Console has been updated to version 3.2.1:
Improved SSO whitelist checks
For a full list of features and fixes see https://github.com/SUSE/stratos/blob/master/CHANGELOG.md#321.
Stratos Metrics has been updated to version 1.2.1:
For a full list of features and fixes see https://github.com/SUSE/stratos-metrics/blob/master/CHANGELOG.md#121.
3.4.2 Features and Fixes #
The
values.yaml
configuration file has undergone significant updates, including changes to the format and name of keys. When migrating from SUSE Cloud Application Platform 1.5.2 to 2.0, the existingvalues.yaml
configuration file (for example,scf-config-values.yaml
) for 1.5.2 cannot be reused for 2.0. A newvalues.yaml
file (for example,kubecf-config-values.yaml
) must be created and used instead. Refer to Section 6, “Appendix: Samplevalues.yaml
File” as a guideline for format and names valid for SUSE Cloud Application Platform 2.0The UAA component is now embedded in KubeCF by default and a separate installation is not necessary.
Fixed
appVersion
field in Chart yaml(s) to reflect the application versionRemoved
sle12
andcflinuxfs2
stack, which were preceded by warnings in CAP 1.5.x releasesRemoved
cf-usb
service broker. Users should use Minibroker or the OSBAPI-compliant brokers provided by the public cloud platforms insteadnfs-broker
is not included in the current releasepxc
is no longer directly taken from upstream; we rely on a docker image (0.9.4) that uses version 5.7.28-31.41Includes these Cloud Foundry component versions:
app-autoscaler: 3.0.0
bits-services: 2.28.0
bosh-dns-aliases: 0.0.3
bpm: 1.1.7
capi: 1.91.0
cf-acceptance-tests: 0.0.13
cf-deployment: 12.36
cf-smoke-tests: 40.0.128
cf-syslog-drain: 10.2.11
cflinuxfs3: 0.167.0
credhub: 2.5.11
diego: 2.44.0
eirini: 0.0.27
garden-runc: 1.19.10
loggregator: 106.3.8
loggregator-agent: 5.3.7
log-cache: 2.6.8
nats: 33
postgres-release: 39
routing: 0.198.0
scf-helper: 1.0.13
silk: 2.28.0
sle15: 10.93
statsd-injector: 1.11.15
sync-integration-tests: 0.0.3
uaa: 74.15.0
Buildpacks:
binary-buildpack: 1.0.36
dotnetcore-buildpack: 2.3.9
go-buildpack: 1.9.11
java-buildpack: 4.29.1
nginx-buildpack: 1.1.7
nodejs-buildpack: 1.7.17
php-buildpack: 4.4.12
python-buildpack: 1.7.12
staticfile-buildpack: 1.5.5
ruby-buildpack: 1.8.15
3.4.3 Known Issues #
Important
The transition from SUSE Cloud Application Platform 1.5.2 to SUSE Cloud Application Platform 2.0 involves a migration of data rather than a direct upgrade. The procedure can be found at https://documentation.suse.com/suse-cap/2.0/single-html/cap-guides/#sec-cap-update
Important
Autoscaler can go into a
CrashLoopBackoff
state if DNS setup is not complete by the time autoscaler comes up. To avoid this situation it is recommended the DNS entries are set up as soon as services (e.g.router-public
) have external IPs assigned to them. Theasactors
pod can also be deleted to recover from this state as kubernetes will re-create the pod
Important
Occasionally, the Autoscaler’s database pod (
asdatabase
) can go into aCrashLoopBackoff
when Autoscaler is enabled via ahelm upgrade
. It is recommended Autoscaler is deployed along with other CAP 2 components during the initialhelm install
.
Important: Mitigating Gorouter DoS Attacks (CVE-2020-15586)
The current release of SUSE Cloud Application Platform is affected by CVE-2020-15586 whereby the Gorouter is vulnerable to a Denial-of-Service (DoS) attack via requests with the "Expect: 100-continue" header. For details regarding this vulnerability, see https://www.cloudfoundry.org/blog/cve-2020-15586/.
If available, operators are advised to upgrade to a SUSE Cloud Application Platform release that is not affected by this vulnerability. Always review the release notes (https://suse.com/releasenotes/) to verify whether a given SUSE Cloud Application Platform release is affected. If it is not possible to upgrade immediately, we recommend operators follow the mitigations from Cloud Foundry’s security update (see https://www.cloudfoundry.org/blog/cve-2020-15586/):
* Configure an HTTP load balancer in front of the Gorouters to drop the Expect 100-continue
header completely.
This may cause delays in HTTP clients that utilize the Expect: 100 continue behavior. However, this should not affect the correctness of HTTP applications.
* Configure an HTTP load balancer in front of the Gorouters to drop the Expect: 100-continue
header and immediately respond with “100 Continue”.
This may cause HTTP clients to send the request body unnecessarily in some cases where the server would have responded with a final status code before requesting the body. However, this should not affect the correctness of HTTP applications.
If you are using a TCP / L4 load balancer for your Gorouters instead of an HTTP load balancer, consider the following: * Add firewall rules to prevent traffic from any source making requests that are causing this panic. ** You may use the extra_headers_to_log property to enable logging of the “Expect” request header to help identify sources of this malicious traffic.
SLE12 and cflinuxfs2 have been removed and are no longer supported. For details regarding the deprecation of these stacks, refer to the previous annoucements at https://www.suse.com/releasenotes/x86_64/SUSE-CAP/1/#sec.1_4_1.issue and https://www.suse.com/releasenotes/x86_64/SUSE-CAP/1/#sec.1_5.issue. Procedures to migrate to new stacks can be found at https://documentation.suse.com/suse-cap/1.5.2/single-html/cap-guides/#id-1.3.4.3.4.
Eirini will only work on a cluster that has
.cluster.local
set as the local domain in the kubelet using ` --cluster-domain` as described at https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#introductionWhen Eirini is enabled, both
features.suse_default_stack
andfeatures.suse_buildpacks
must be enabled. A cflinuxfs3 Eirini image is currently not available, and the SUSE stack must be used.On subsequent deployments with Eirini enabled, deployments may result in the
bits
pod going into aCrashLoopBackoff
state with atls: private key does not match public key
error. This occurs after an initial deployment wiht Eirini enabled is not properly cleaned up. The csr forbits
is not namespaced and will not be removed whenhelm delete kubecf
is performed and must be deleted manually usingkubectl delete csr kubecf-bits-service-ssl
before another deployment is made.The Open Service Broker for Azure is only compatible with Kubernetes 1.15 or earlier.
The
cf-usb
service brokers from CAP 1.x will not work with CAP 2.0, along with Kubernetes 1.16 or higher, so they are no longer supported. As noted in the Features & Fixes above, please migrate over to either Minibroker or an OSBAPI-compliant broker available via your public cloud platform.During the Open Service Broker for Azure set up process, the svc/catalog chart install will encounter a
OOMKilled
state and fail. The controllerManager’s requests and limits for the CPU and memory must be increased to avoid this. As an example, increasing these values to double the default will allow for a successful installation.helm install catalog svc-cat/catalog \ --namespace catalog \ --set controllerManager.healthcheck.enabled=false \ --set apiserver.healthcheck.enabled=false \ --set controllerManager.resources.requests.cpu=200m \ --set controllerManager.resources.requests.memory=40Mi \ --set controllerManager.resources.limits.cpu=200m \ --set controllerManager.resources.limits.memory=40Mi
4 Obtaining Source Code #
This SUSE product includes materials licensed to SUSE under the GNU General Public License (GPL). The GPL requires SUSE to provide the source code that corresponds to the GPL-licensed material. The source code is available for download at https://www.suse.com/download-linux/source-code.html. Also, for up to three years after distribution of the SUSE product, upon request, SUSE will mail a copy of the source code. Requests should be sent by e-mail to sle_source_request@suse.com or as otherwise instructed at https://www.suse.com/download-linux/source-code.html. SUSE may charge a reasonable fee to recover distribution costs.
5 Legal Notices #
SUSE makes no representations or warranties with regard to the contents or use of this documentation, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, SUSE reserves the right to revise this publication and to make changes to its content, at any time, without the obligation to notify any person or entity of such revisions or changes.
Further, SUSE makes no representations or warranties with regard to any software, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, SUSE reserves the right to make changes to any and all parts of SUSE software, at any time, without any obligation to notify any person or entity of such changes.
Any products or technical information provided under this Agreement may be subject to U.S. export controls and the trade laws of other countries. You agree to comply with all export control regulations and to obtain any required licenses or classifications to export, re-export, or import deliverables. You agree not to export or re-export to entities on the current U.S. export exclusion lists or to any embargoed or terrorist countries as specified in U.S. export laws. You agree to not use deliverables for prohibited nuclear, missile, or chemical/biological weaponry end uses. Refer to https://www.suse.com/company/legal/ for more information on exporting SUSE software. SUSE assumes no responsibility for your failure to obtain any necessary export approvals.
Copyright © 2017-2021 SUSE LLC.
This release notes document is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (CC-BY-SA-4.0). You should have received a copy of the license along with this document. If not, see https://creativecommons.org/licenses/by-nd/4.0/.
SUSE has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at https://www.suse.com/company/legal/ and one or more additional patents or pending patent applications in the U.S. and other countries.
For SUSE trademarks, see SUSE Trademark and Service Mark list (https://www.suse.com/company/legal/). All third-party trademarks are the property of their respective owners.
6 Appendix: Sample values.yaml
File #
# REQUIRED: the domain that the deployment will be visible to the user.
system_domain: ~
# Set or override job properties. The first level of the map is the instance group name. The second
# level of the map is the job name. E.g.:
# properties:
# adapter:
# adapter:
# scalablesyslog:
# adapter:
# logs:
# addr: kubecf-log-api:8082
#
properties: {}
credentials: {}
variables: {}
kube:
# The storage class to be used for the instance groups that need it (e.g. bits, database and
# singleton-blobstore). If it's not set, the default storage class will be used.
storage_class: ~
# The psp key contains the configuration related to Pod Security Policies. By default, a PSP will
# be generated with the necessary permissions for running KubeCF. To pass an existing PSP and
# prevent KubeCF from creating a new one, set the kube.psp.default with the PSP name.
psp:
default: ~
releases:
# The defaults for all releases, where we do not otherwise override them.
defaults:
url: registry.suse.com/cap
stemcell:
os: SLE_15_SP1
version: 23.21-7.0.0_374.gb8e8e6af
app-autoscaler:
version: 3.0.0
bits-service:
version: 2.28.0
brain-tests:
version: v0.0.12
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
cf-acceptance-tests:
version: 0.0.13
stemcell:
os: SLE_15_SP1
version: 23.21-7.0.0_374.gb8e8e6af
cf-smoke-tests:
version: 40.0.128
stemcell:
os: SLE_15_SP1
version: 25.2-7.0.0_374.gb8e8e6af
# pxc is not a BOSH release.
pxc:
image:
repository: registry.suse.com/cap/pxc
tag: 0.9.4
eirini:
version: 0.0.27
stemcell:
os: SLE_15_SP1
version: 23.21-7.0.0_374.gb8e8e6af
postgres:
version: "39"
sle15:
version: "10.93"
sync-integration-tests:
version: v0.0.3
suse-staticfile-buildpack:
url: registry.suse.com/cap
version: "1.5.5.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-staticfile-buildpack/packages/staticfile-buildpack-sle15/staticfile-buildpack-sle15-v1.5.5.1-5.1-eaf36a02.zip
suse-java-buildpack:
url: registry.suse.com/cap
version: "4.29.1.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-java-buildpack/packages/java-buildpack-sle15/java-buildpack-sle15-v4.29.1.1-543ec059.zip
suse-ruby-buildpack:
url: registry.suse.com/cap
version: "1.8.15.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-ruby-buildpack/packages/ruby-buildpack-sle15/ruby-buildpack-sle15-v1.8.15.1-4.1-2b6d6879.zip
suse-dotnet-core-buildpack:
url: registry.suse.com/cap
version: "2.3.9.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-dotnet-core-buildpack/packages/dotnet-core-buildpack-sle15/dotnet-core-buildpack-sle15-v2.3.9.1-1.1-e74bd89e.zip
suse-nodejs-buildpack:
url: registry.suse.com/cap
version: "1.7.17.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-nodejs-buildpack/packages/nodejs-buildpack-sle15/nodejs-buildpack-sle15-v1.7.17.1-1.1-7e96d2dd.zip
suse-go-buildpack:
url: registry.suse.com/cap
version: "1.9.11.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-go-buildpack/packages/go-buildpack-sle15/go-buildpack-sle15-v1.9.11.1-2.1-d5c02636.zip
suse-python-buildpack:
url: registry.suse.com/cap
version: "1.7.12.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-python-buildpack/packages/python-buildpack-sle15/python-buildpack-sle15-v1.7.12.1-2.1-ebd0f50d.zip
suse-php-buildpack:
url: registry.suse.com/cap
version: "4.4.12.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-php-buildpack/packages/php-buildpack-sle15/php-buildpack-sle15-v4.4.12.1-4.1-2c4591cb.zip
suse-nginx-buildpack:
url: registry.suse.com/cap
version: "1.1.7.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-nginx-buildpack/packages/nginx-buildpack-sle15/nginx-buildpack-sle15-v1.1.7.1-1.1-fbf90d1f.zip
suse-binary-buildpack:
url: registry.suse.com/cap
version: "1.0.36.1"
stemcell:
os: SLE_15_SP1
version: 25.1-7.0.0_374.gb8e8e6af
file: suse-binary-buildpack/packages/binary-buildpack-sle15/binary-buildpack-sle15-v1.0.36.1-1.1-37ec2cbf.zip
multi_az: false
high_availability: false
# Sizing takes precedence over the high_availability property. I.e. setting the instance count
# for an instance group greater than 1 will make it highly available.
sizing:
adapter:
instances: ~
api:
instances: ~
asactors:
instances: ~
asapi:
instances: ~
asmetrics:
instances: ~
asnozzle:
instances: ~
auctioneer:
instances: ~
bits:
instances: ~
cc_worker:
instances: ~
credhub:
instances: ~
database:
instances: ~
persistence:
size: 20Gi
diego_api:
instances: ~
diego_cell:
ephemeral_disk:
# Size of the ephemeral disk used to store applications in MB
size: 40960
# The name of the storage class used for the ephemeral disk PVC.
storage_class: ~
instances: ~
doppler:
instances: ~
eirini:
instances: ~
log_api:
instances: ~
nats:
instances: ~
router:
instances: ~
routing_api:
instances: ~
scheduler:
instances: ~
uaa:
instances: ~
tcp_router:
instances: ~
# External endpoints are created for the instance groups only if features.ingress.enabled is false.
services:
router:
annotations: ~
type: LoadBalancer
externalIPs: []
clusterIP: ~
ssh-proxy:
annotations: ~
type: LoadBalancer
externalIPs: []
clusterIP: ~
tcp-router:
annotations: ~
type: LoadBalancer
externalIPs: []
clusterIP: ~
port_range:
start: 20000
end: 20008
settings:
router:
# tls sets up the public TLS for the router. The tls keys:
# crt: the certificate in the PEM format. Required.
# key: the private key in the PEM format. Required.
tls: {}
# crt: |
# -----BEGIN CERTIFICATE-----
# ...
# -----END CERTIFICATE-----
# key: |
# -----BEGIN PRIVATE KEY-----
# ...
# -----END PRIVATE KEY-----
features:
eirini:
# When eirini is enabled, both suse_default_stack and suse_buildpacks must be enabled as well.
enabled: false
registry:
service:
# This setting is not currently configurable and must be HIDDEN
nodePort: 31666
ingress:
enabled: false
tls:
crt: ~
key: ~
annotations: {}
labels: {}
suse_default_stack:
enabled: true
suse_buildpacks:
enabled: true
autoscaler:
enabled: false
credhub:
enabled: true
# Disabling routing_api will also disable the tcp_router instance_group
routing_api:
enabled: true
# embedded_database enables the embedded PXC sub-chart. Disabling it allows using an external, already seeded,
embedded_database:
enabled: true
blobstore:
# Possible values for provider: singleton and s3.
provider: singleton
s3:
aws_region: ~
blobstore_access_key_id: ~
blobstore_secret_access_key: ~
blobstore_admin_users_password: ~
# The following values are used as S3 bucket names.
app_package_directory_key: ~
buildpack_directory_key: ~
droplet_directory_key: ~
resource_directory_key: ~
# The external database type can be either 'mysql' or 'postgres'.
external_database:
enabled: false
require_ssl: false
ca_cert: ~
type: ~
host: ~
port: ~
databases:
uaa:
name: uaa
password: ~
username: ~
cc:
name: cloud_controller
password: ~
username: ~
bbs:
name: diego
password: ~
username: ~
routing_api:
name: routing-api
password: ~
username: ~
policy_server:
name: network_policy
password: ~
username: ~
silk_controller:
name: network_connectivity
password: ~
username: ~
locket:
name: locket
password: ~
username: ~
credhub:
name: credhub
password: ~
username: ~
# Enable or disable instance groups for the different test suites.
# Only smoke tests should be run in production environments.
#
# __ATTENTION__: The brain tests do things with the cluster which
# required them to have `cluster-admin` permissions (i.e. root).
# Enabling them is thus potentially insecure. They should only be
# activated for isolated testing.
testing:
brain_tests:
enabled: false
cf_acceptance_tests:
enabled: false
smoke_tests:
enabled: true
sync_integration_tests:
enabled: false
ccdb:
encryption:
rotation:
# Key labels must be <= 240 characters long.
key_labels:
- encryption_key_0
current_key_label: encryption_key_0
operations:
# A list of configmap names that should be applied to the BOSH manifest.
custom: []
# Inlined operations that get into generated ConfigMaps. E.g. adding a password variable:
# operations:
# inline:
# - type: replace
# path: /variables/-
# value:
# name: my_password
# type: password
inline: []
k8s-host-url: ""
k8s-service-token: ""
k8s-service-username: ""
k8s-node-ca: ""
eirini:
global:
labels: {}
annotations: {}
env:
# This setting is not configurable and must be HIDDEN from the user.
# It's a workaround to replace the port eirini uses for the registry
DOMAIN: '127.0.0.1.nip.io:31666" #'
services:
loadbalanced: true
opi:
image_tag: "1.5.0"
image: registry.suse.com/cap/opi
metrics_collector_image: registry.suse.com/cap/metrics-collector
bits_waiter_image: registry.suse.com/cap/bits-waiter
route_collector_image: registry.suse.com/cap/route-collector
route_pod_informer_image: registry.suse.com/cap/route-pod-informer
route_statefulset_informer_image: registry.suse.com/cap/route-statefulset-informer
event_reporter_image: registry.suse.com/cap/event-reporter
event_reporter_image_tag: "1.5.0"
staging_reporter_image: registry.suse.com/cap/staging-reporter
staging_reporter_image_tag: "1.5.0"
#
registry_secret_name: eirini-registry-credentials
namespace: eirini
kubecf:
enable: false
use_registry_ingress: false
ingress_endpoint: ~
kube:
external_ips: []
deny_app_ingress: false
cc_api:
serviceName: "api"
staging:
downloader_image: registry.suse.com/cap/recipe-downloader
downloader_image_tag: "1.5.0-24.1"
executor_image: registry.suse.com/cap/recipe-executor
executor_image_tag: "1.5.0-24.1"
uploader_image: registry.suse.com/cap/recipe-uploader
uploader_image_tag: "1.5.0-24.1"
enable: true
tls:
client:
secretName: "var-eirini-tls-client-cert"
certPath: "certificate"
keyPath: "private_key"
cc_uploader:
secretName: "var-cc-bridge-cc-uploader"
certPath: "certificate"
keyPath: "private_key"
ca:
secretName: "var-eirini-tls-client-cert"
path: "ca"
stagingReporter:
secretName: "var-eirini-tls-client-cert"
certPath: "certificate"
keyPath: "private_key"
caPath: "ca"
tls:
opiCapiClient:
secretName: "var-eirini-tls-client-cert"
keyPath: "private_key"
certPath: "certificate"
opiServer:
secretName: "var-eirini-tls-server-cert"
certPath: "certificate"
keyPath: "private_key"
capi:
secretName: "var-eirini-tls-server-cert"
caPath: "ca"
eirini:
secretName: "var-eirini-tls-server-cert"
caPath: "ca"
events:
enable: true
# All configs in this section should be HIDDEN from the user; they are
# here to adapt the Eirini helm chart for KubeCF use.
tls:
capiClient:
secretName: "var-cc-tls"
keyPath: "private_key"
certPath: "certificate"
capi:
secretName: "var-cc-tls"
caPath: "ca"
logs:
# disable fluentd, use eirinix-loggregator-bridge (HIDDEN from the user).
enable: false
# HIDDEN from the user as changing this breaks logging.
serviceName: doppler
# All configs in this section should be HIDDEN from the user; they are here
# to adapt the Eirini helm chart for KubeCF use.
metrics:
enable: true
tls:
client:
secretName: "var-loggregator-tls-doppler"
keyPath: "private_key"
certPath: "certificate"
server:
secretName: "var-loggregator-tls-doppler"
caPath: "ca"
rootfsPatcher:
enable: false
timeout: 2m
# All configs in this section should be HIDDEN from the user; they are here
# to adapt the Eirini helm chart for KubeCF use.
routing:
enable: true
nats:
secretName: "var-nats-password"
passwordPath: "password"
serviceName: "nats"
secretSmuggler:
enable: false
bits:
download_eirinifs: false
global:
labels: {}
annotations: {}
images:
bits_service: registry.suse.com/cap/bits-service:bits-1.0.15-15.1.6.2.220-24.2
env:
# This setting is not configurable and must be HIDDEN from the user.
DOMAIN: 127.0.0.1.nip.io
ingress:
endpoint: ~
use: false
kube:
external_ips: []
services:
loadbalanced: true
blobstore:
serviceName: "singleton-blobstore"
userName: "blobstore-user"
secret:
name: "var-blobstore-admin-users-password"
passwordPath: "password"
secrets:
BITS_SERVICE_SECRET: "secret"
BITS_SERVICE_SIGNING_USER_PASSWORD: "notpassword123"
useExistingSecret: true
tls_secret_name: bits-service-ssl
tls_cert_name: certificate
tls_key_name: private_key
tls_ca_name: ca
eirinix:
persi-broker:
service-plans:
- id: default
name: "default"
description: "Existing default storage class"
kube_storage_class: "default"
free: true
default_size: "1Gi"