Running the Service
This document provides instructions on how to run the ros_llm
service node and interact with it.
How to Run the Service
To run the chat service, you first need to source your ROS2 environment and then use the ros2 run
command.
You can run the service node with the following command. This will start the LLM client and make it available for receiving prompts.
Arguments
You can customize the behavior of the node by passing the following ROS parameters.
Argument | Description | Default |
---|---|---|
server_url |
The URL of the Llama.CPP server. | http://127.0.0.1:5001/v1/chat/completions |
temperature |
The sampling temperature. A lower value makes the output more deterministic, while a higher value makes it more creative. | 1.0 |
n_predict |
The number of tokens to predict. A smaller value results in shorter answers. Use -1 to disable this limit. |
-1 (Disabled) |
history_max_length |
The maximum number of previous prompts to keep in the context, defining the "memory" of the assistant. | 10 |
system_prompt |
The system prompt that guides the assistant's behavior. | 'You are a helpful robot assistant.' |
rag_enabled |
Unimplemented. Toggles Retrieval-Augmented Generation (RAG) mode. | false |
rag_system_prompt_template |
The system prompt template used for RAG. The {context} placeholder will be replaced with the retrieved information. |
"Use the following context to answer the user's question. If the context doesn't contain the answer, say you don't know.\n\n--- Context ---\n{context}\n--- End Context ---" |
Service Requests
LLM
To send a prompt to the LLM, you can call the /llm
service.
It uses the ric_messages/srv/LLMInput
service type. Replace ${prompt}
with your desired prompt.
Clear History
To clear the conversation history maintained by the node, you can call the /clear_history
service. This is useful for starting a fresh conversation without restarting the node.
It uses the std_srvs/srv/Empty
service type.