Troubleshoot deployments to OpenShift clusters

If you run into issues during the installation of MOSTLY AI, you can review the listed errors below and see if they match your issue. Each issue includes a description of the problem and solution that shows how to overcome or workaround the issue.

Error: would violate PodSecurity allowPrivilegeEscalation

The installation error would violate PodSecurity allowPrivilegeEscalation might indicate misconfigurations in the values.yaml file.

Problem

When you run the helm command to install MOSTLY AI on an OpenShift cluster, you might see the installation error similar to the one below.

W0714 14:04:39.656905   47432 warnings.go:70] would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "mostly-rabbitmq" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "mostly-rabbitmq" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "mostly-rabbitmq" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "mostly-rabbitmq" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0714 14:04:39.711306   47432 warnings.go:70] would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "mostly-psql" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "mostly-psql" must set securityContext.capabilities.drop=["ALL"])

Solution

• Verify that you have the correct storageClassName defined for the CORDINATOR: pvc: storageClassName key. To check your OpenShift storageClassNames, open your OpenShift console, and navigate to Storage > StorageClasses. The name defined for CORDINATOR: pvc: storageClassName must match one of the entires.
• Verify that you the platform key is set to ocp and not k8s.
  values.yaml

  platform: k8s # incorrect
  platform: ocp # correct

Error: UPGRADE FAILED: failed to create resource: Secret in version "v1" cannot be handled as a Secret

The error UPGRADE FAILED: failed to create resource: Secret in version "v1" cannot be handled as a Secret might indicate an incorrect or missing secret key in the docker_secret key in the values.yaml file.

Problem

When you run the helm command to deploy MOSTLY AI in an OpenShift cluster, you might see the deployment error similar to the one below.

UPGRADE FAILED: failed to create resource: Secret in version "v1" cannot be handled as a Secret: illegal base64 data at input byte 6

Make sure that you have set the Docker pull image secret in docker_secret.

Solution

• Double-check that you have the correct secret key in docker_secret.
• Make sure that the secret key value is not wrapped in quotation marks.

If the suggestions above do not help to resolve the issue, contact your Customer Success Engineer.

Error: UPGRADE FAILED: cannot patch "mostly-data" with kind PersistentVolumeClaim

The error UPGRADE FAILED: cannot patch "mostly-data" with kind PersistentVolumeClaim might indicate intermittent installation issues.

Problem

When you run the helm command to deploy MOSTLY AI in an OpenShift cluster, you might see an installation error similar to the one below.

Error: UPGRADE FAILED: cannot patch "mostly-data" with kind PersistentVolumeClaim: PersistentVolumeClaim "mostly-data" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	... // 2 identical fields
  	Resources:        {Requests: {s"storage": {i: {...}, s: "50Gi", Format: "BinarySI"}}},
  	VolumeName:       "",
- 	StorageClassName: &"efs-sc",
+ 	StorageClassName: &"efs",
  	VolumeMode:       &"Filesystem",
  	DataSource:       nil,
  	DataSourceRef:    nil,
  }

Solution

You can work around the issue above by removing the mostly-ai project from your OpenShift cluster, recreating it, and running the installation again.

Steps

1. Delete the the mostly-ai project from your OpenShift cluster.

   oc delete namespace mostly-ai
2. Create the mostly-ai project again.

   oc new-project mostly-ai
3. Run the installation again with the helm command.

   helm upgrade --install mostly-ai ./ --values values.yaml --namespace mostly-ai