This manual helps system administrators with diagnosing and troubleshooting runtime issues that may occur when running MOSTLY AI.

It describes MOSTLY AI’s architecture and its components, provides the locations of all configuration files, and lists out the commands to check health statuses and retrieve logs.

MOSTLY AI’s components

Below is the architecture diagram of MOSTLY AI 2.2. The yellow numbers in this diagram refer to further details about the containers that you can find in the table further down below.

MOSTLY AI’s architecture
Container and image name Description Container Lifecycle

Application VM(s)

1

mostly-app
mostlyai/mostly-app-enterprise

Contains MOSTLY AI’s frontend, backend and the API. Generally reachable over port 8080.

Always Up

2

mostly-coordinator
mostlyai/mostly-coordinator

This component takes all requests from the web application and coordinates execution of tasks on the main AI engine.

Always Up

3

mostly-keycloak
mostlyai/mostly-keycloak

Keycloak is an open-source identity management, authentication, and authorization tool. This container has a pre-configured Keycloak instance for MOSTLY AI.

Always Up

4

mostly-rabbitmq
rabbitmq:3.8.14

Message queue handling communications between the AI engine and the application.

Always Up

5

mostly-db
postgres:11

Database instance of the system. Has databases for app, coordinator and Keycloak.

Always Up

Workload VM(s)

6

mostly-agent
mostlyai/mostly-agent-cpu

MOSTLY AI’s AI engine.

Always Up

7

mostly-engine-v2
mostlyai/mostly-engine

The main engine component, which does the AI training and data generation.

On Demand

8

mostly-analyze
mostlyai/mostly-analyze

Engine component, which creates the Quality Assurance report for privacy and accuracy of the generated data versus the original data.

On Demand

9

mostly-db-connector
mostlyai/mostly-db-connector

Engine component responsible for the database operations.

On Demand

10

mostly-smars
mostlyai/mostly-smars

Engine component responsible for the Smart Select algorithm.

On Demand

Using the command-line utilities

MOSTLY AI comes with a couple of command line utilites called mostly-ai and mostly-agent, that are useful for conducting basic operations on the system.

They are already in the default path, but can be located in their respective folders:
/home/<user>/<installation_folder>/mostly-ai/bin
/home/<user>/<installation_folder>/mostly-agent/bin

You can use it to start, stop, or check MOSTLY AI’s status:

Start

mostly-ai start
Stop

mostly-ai stop
Check status

mostly-ai status

and

Start

mostly-agent start
Stop

mostly-agent stop
Check status

mostly-agent status

For the full list of commands, execute mostly-ai --help or mostly-agent --help respectively.

Configuration file locations

The following locations contain MOSTLY AI’s configuration files and working directory. You can find them in the installation folder, which is either /home/<user>/<installation_folder> or /opt/<installation_folder>.

Folder Location Description

etc

<installation_folder>/mostly-ai/etc

Configuration files

data

<installation_folder>/mostly-ai/data

Working directory

You can find the following configuration files in the etc folder:

Folder Description

docker-compose.yml

Contains container definitions, their relationships and integrations.

.env (hidden file)

Contains docker-compose.yml‘s environment variables.

realm.yml

Contains the application’s start-up configuration.

Advanced troubleshooting

The commands below can help diagnose issues. Prefix them with sudo if the required privileges to run them are missing.

Container health status

Command

docker ps -all

Notes

Lists all the containers, when they were created and their status. If there is a missing container (see list of containers above), this indicates that it has stopped.

Also, if a container is constantly restarting (status always resetting to “Up x seconds”), this is an indication of a failure.

In such cases, stopping the application and restarting it using mostly-ai commands will help. If the issue persists, checking the logs is recommended.

Container logs

Command

docker logs <container_name>
e.g., docker logs mostly-agent

docker logs <container_id> e.g., docker logs 5c29f59982b9

Notes

Displays the logs of a container to the console.