The Dockerfile
Info
This section assumes that you have basic knowledge about Dockerfiles. If you don't have basic knowledge about Dockerfiles, please read the Docker documentation first and then come back.
The Base Image
The Dockerfile uses Ubuntu 22.04 as its base image.
Arguments
When you first open the Dockerfile, you'll notice a bunch of ARG params. Those are arguments that you can pass in the compose.yml. The relevant ones are listed here:
| Argument Name | Description |
|---|---|
OLLAMA_PORT |
The port for the Ollama service |
HTTP_PORT |
The port for the HTTP Server |
LLM_CHAT_HISTORY_LIMIT |
How many messages should be allowed in the chat history until it gets cleaned up |
Environment Variables
Next, you'll see the environment variables we use. Note that the ones we use in the executable itself are prefixed with HRIC_.
| Environment Variable | Description |
|---|---|
APT_CACHE |
The APT cache mount. Where we save our downloaded APT packages to make the next rebuild take less time. Changing this may break the cache mount feature. Therefore, it is adviced to not change this. |
OLLAMA_CACHE |
Where Ollama should be downloaded. You can change this value at will. If Ollama is located here already, it will not be re-downloaded, but copied to the right location instead. |
BUILD |
Where the project files will end up. If you set this value, you should also change the volumes in the compose.yml. |
HRIC_OLLAMA_PORT |
The port for the Ollama service. |
HRIC_HTTP_PORT |
The port for the HTTP server. |
HRIC_VENV_NAME |
The name of the conda environment that we'd like the LLM-Server to run in. |
HRIC_LLM_CHAT_HISTORY_LIMIT |
How many messages should be allowed in the chat history until it gets cleaned up |
Python
We use Python 3.10 for the container. We also use pip as the package manager for Python libraries. And finally, we use Miniconda. However, there are seperate python environments in the miniconda virtual environment.
The HTTP Library
We use Flask for the python implementation of our HTTP endpoints. To expose those for production, we use Waitress.
Tests
We use pytest with coverage to test our Python code.
The start scripts
Once everything is built, the docker_setup/docker-start.sh is called. It is here where we copy the checkpoints we downloaded in the container into the project files. We also start the ollama service here.