Skip to content

Running the Service

This document provides instructions on how to run the ros_llm service node and interact with it.

How to Run the Service

To run the chat service, you first need to source your ROS2 environment and then use the ros2 run command.

You can run the service node with the following command. This will start the LLM client and make it available for receiving prompts.

# Source your ROS2 workspace
source install/setup.bash

ros2 run llm llm_node

Arguments

You can customize the behavior of the node by passing the following ROS parameters.

ros2 run llm llm_node --ros-args -p <argument>:=<value>
Argument Description Default
server_url The URL of the Llama.CPP server. http://127.0.0.1:5001/v1/chat/completions
temperature The sampling temperature. A lower value makes the output more deterministic, while a higher value makes it more creative. 1.0
n_predict The number of tokens to predict. A smaller value results in shorter answers. Use -1 to disable this limit. -1 (Disabled)
history_max_length The maximum number of previous prompts to keep in the context, defining the "memory" of the assistant. 10
system_prompt The system prompt that guides the assistant's behavior. 'You are a helpful robot assistant.'
rag_enabled Unimplemented. Toggles Retrieval-Augmented Generation (RAG) mode. false
rag_system_prompt_template The system prompt template used for RAG. The {context} placeholder will be replaced with the retrieved information. "Use the following context to answer the user's question. If the context doesn't contain the answer, say you don't know.\n\n--- Context ---\n{context}\n--- End Context ---"

Service Requests

LLM

To send a prompt to the LLM, you can call the /llm service.

It uses the ric_messages/srv/LLMInput service type. Replace ${prompt} with your desired prompt.

ros2 service call /llm ric_messages/srv/LLMInput "{'prompt': '${prompt}'}"

Clear History

To clear the conversation history maintained by the node, you can call the /clear_history service. This is useful for starting a fresh conversation without restarting the node.

It uses the std_srvs/srv/Empty service type.

ros2 service call /clear_history std_srvs/srv/Empty