Troubleshoot minikube deployment issues

Learn how you can troubleshoot various issues when you deploy MOSTLY AI to a minikube cluster. Each of the listed issues includes a description of the problem and solution that shows how to workaround the issue.

Cannot upload files

Problem

When you attempt to upload files in MOSTLY AI, you might see the error notification Error while connecting API.

Solution

1. Check if you have enabled the ingress add-on in your minikube cluster. For details, see Task 10: Install required minikube addons .

2. Make sure that you have enabled the ingress annotations for the mostly-app and mostly-keycloak services in the values.yaml file as listed below.

   values.yaml

   mostly-app:
       ingress:
           annotations:
             nginx.ingress.kubernetes.io/proxy-body-size: 10240m
             nginx.ingress.kubernetes.io/proxy-buffer-size: 128k
             nginx.org/proxy-connect-timeout: 3000s
             nginx.org/proxy-read-timeout: 3000s
             nginx.org/client-max-body-size: 3000m
           # annotations: {}
   values.yaml

   mostly-keycloak:
   ...
       ingress:
           annotations:
               nginx.ingress.kubernetes.io/proxy-body-size: 10240m
               nginx.ingress.kubernetes.io/proxy-buffer-size: 128k
           # annotations: {}

3. Apply the changes to the values.yaml file and redeploy the MOSTLY AI Helm chart.

Generator is stuck in Queued status

After you deploy MOSTLY AI in a minikube cluster, your first sanity test can be to train a new generator. However, the generator training might make no progress and remain in the Queued status indefinitely.

Problem

After starting the training of a new generator, its status is Queued and it remains as such indefinitely without making progress.

Cause

The most likely cause for this is that in your values.yaml file, all CPU and memory resources available to the minikube cluster have been allocated to the Default compute.

Solution

Reallocate the Default compute resources to be lower than the total resources allocated for your minikube cluster. For example, if your minikube cluster has 14 CPUs and 24 GB of memory, allocate 10 CPUs and 20 GB of memory to the Default compute.

1. In the values.yaml file, edit the defaultComputePool section for the mostly-app service.
   values.yaml

   mostly-app:
     deployment: 
       ...
       mostly:
         defaultComputePool:
           name: Default
           type: KUBERNETES
           toleration: engine-jobs
           resources:
             cpu: 10
             memory: 20
             gpu: 0
2. Save the values.yaml file.
3. Remove your current deployment by deleting the mostly-ai namespace.

   kubectl delete namespace mostly-ai
4. Re-deploy MOSTLY AI.

   helm upgrade --install mostly-ai ./mostly-combined --values ocp-values.yaml --namespace mostly-ai --create-namespace