CPU and memory profiling
This document (000020887) is provided subject to the disclaimer at the end of this document.
Situation
Resolution
In general, these are the steps to perform along with capturing profiles.
- "kubectl top pod" results
- "ps aux" results inside the pod, "kubectl exec ......... -- ps aux ..."
- capture the memory/cpu profiles (using the instructions detailed below)
- collect the pod logs
Below are steps on how to perform such profile captures.
Total Profile Capture (v4.0.2+)
Performance issues can be caused by various factors. To have the whole picture, the below steps will capture all profiles from each of the NeuVector components.
kubectl -n neuvector exec -it neuvector-manager-pod-name -- cli #neuvector-svc-controller.neuvector> login admin#neuvector-svc-controller.neuvector> show controller admin#neuvector-svc-controller.neuvector> request controller <controller_id> profile -c all admin#neuvector-svc-controller.neuvector> show enforcer admin#neuvector-svc-controller.neuvector> request enforcer <enforcer_id> profile -c all admin#neuvector-svc-controller.neuvector> exit
The default profiling duration is 30s. So after 35 seconds, the profiles should be available to copy from each NeuVector controller and enforcer pods under the /var/neuvector/profile folder.
Controller profiles
- ctl.memory.prof
- ctl.gc.memory.prof
- ctl.cpu.prof
- ctl.goroutine.prof
Enforcer profiles
- enf.memory.prof
- enf.gc.memory.prof
- enf.cpu.prof
- enf.goroutine.prof
Copy the profile files from the controller and enforcer pods.
for i in `kubectl get pods -n neuvector | egrep "controller|enforcer" | awk '{print $1}'`;do kubectl -n neuvector cp $i:/var/neuvector/profile $i.profile; done
Controller | Enforcer Memory Profile
In case only specific profile capture is needed, the below steps demonstrate that. Repeat the request command for each controller or enforcer.
kubectl -n neuvector exec -it $Manager_Pod -- cli #neuvector-svc-controller.neuvector> login admin#neuvector-svc-controller.neuvector> show (controller|enforcer) admin#neuvector-svc-controller.neuvector> request (controller|enforcer) ($Controller_id|$Enforcer_id) profile -c (cpu|memory) -d 30 admin#neuvector-svc-controller.neuvector> exit
Copy the profile files from pods.
kubectl -n neuvector cp ($Controller_pod|$Enforcer_pod):/var/neuvector/profile/(enf|ctl).(cpu|memory).prof ($Controller_pod|$Enforcer_pod).(cpu|memory).prof
NOTE: Update the OR options example above. For example (controller|enforcer) suggest using either controller or enforcer in its place.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020887
- Creation Date: 13-Dec-2022
- Modified Date:11-Jan-2023
-
- SUSE NeuVector
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com