Skip to content

Running with Docker

This project can be run using Docker and Docker Compose. Install it from here if not already available.

IMPORTANT: Make sure to also clone the ric-messages git submodule located in src folder with:

git submodule update --init

Running

Start the required ROS2 nodes as this node waits on them to become available: ros_stt, ros_llm and ros_tts.

You can run the ROS2 chat node using:

docker compose up -d

This will first build or pull the required docker images and start the chat ROS2 node that provides the /chat service.

Important: Do note that the ROS2 node makes use of rmw_zenoh for ROS2 communication. Use the provided zenoh_router for this purpose.

Send ROS2 messages according to the usage below.

Manual

Follow the relevant ROS2 documentation on how to build packages and run nodes.

Note: The recommended way to build the ROS2 node as of writing is using colcon.

Services

chat-node

This service is responsible for running the ROS2 chat node, providing the interface for the whole chat pipeline including STT, LLM and TTS.

  • Uses harbor.hb.dfki.de/helloric/ros_chat:latest (VPN required) or builds from the local Dockerfile
  • Provides a ROS2 service at /chat which
  • Sends audio bytes to the ros_stt node and receives the transcribed audio as well as the language as language code (see AudioBytesToText.srv in ric-messages).
  • Sends the transcribed audio from ros_stt to ros_llm and receives the response as string (see LLMChat.srv in ric-messages).
  • Sends the response from ros_llm and the language from ros_stt to ros_tts and receives audio bytes back (see TextToAudioBytes.srv in ric-messages).
  • The response consists of the language code from ros_stt, the response and extracted emotion from ros_llm and the audio bytes from ros_tts (see Chat.srv in ric-messages).
  • It uses Zenoh as RMW implementation by default. To change it, refer to the zenoh_router documentation.

Environment

Key Description Value
ROS_AUTOMATIC_DISCOVERY_RANGE Disables automatic discovery in ROS2. OFF
RMW_IMPLEMENTATION ROS2 middleware implementation. rmw_zenoh_cpp
ZENOH_ROUTER_CHECK_ATTEMPTS Number of attempts to check for Zenoh router. 0 means wait indefinitely. 0
ZENOH_CONFIG_OVERRIDE Zenoh configuration override, see rmw_zenoh. mode="client";connect/endpoints=["tcp/host.docker.internal:7447"]
PYTHONUNBUFFERED Prevents Python from buffering stdout and stderr. 1

Usage

Create a ROS2 client for the /chat service and call it. The service uses the ric_messages/srv/Chat interface. For usage examples, check out service.