The Chat History

What is a chat?

We manage the Chat History outside the Prompt Manager, because the chats are not part of a system prompt and we need those chats to be a bit more dynamic.

Essentially, this chat consists of two roles: The user and the assistant.

For the LLM, the example would look like this:

{
  "model": "<...>"
  "messages": [
    {"role": "system", "content": "<The System Prompt>"},
    {"role": "user", "content": "Hey, what is 1 + 1?"},
    {"role": "assistant", "content": "It's two, obviously."},
    {"role": "user", "content": "Can you provide me the doubled value of the previous answer?"},
    {"role": "assistant", "content": "That would be four."}
  ]
  ...
}

On providing another request, we'd just add the next question onto the messages array:

{
  "model": "<...>"
  "messages": [
    {"role": "system", "content": "<The System Prompt>"},
    {"role": "user", "content": "Hey, what is 1 + 1?"},
    {"role": "assistant", "content": "It's two, obviously."},
    {"role": "user", "content": "Can you provide me the doubled value of the previous answer?"},
    {"role": "assistant", "content": "That would be four."},
    {"role": "user", "content": "Now halve it again."}
  ]
  ...
}

Too much information!

To prevent the LLM from hallucinating (providing nonsensical answers), we need to limit the chat history a bit. Which is why we provide some threshold as an environment variable: HRIC_LLM_CHAT_HISTORY_LIMIT, which can be updated within the compose (see this part of the documentation). Currently, we just clear every single message in the history

Trading out histories

The benefit of this approach is that we can just have different chat histories without trashing our main history or confusing our LLM.

The main chat history is stored within the LLM class. Then we also have a fail chat history that is dynamically created upon every request and deleted afterwards.

To use another history, you just have to call the LLM with another parameter:

from llm.chat_history import ChatHistory
from llm.llm import LLM

fake_history = ChatHistory()
fake_history.chat_as_user('Hello, we are best friends, right?')
fake_history.chat_as_llm('But of course we are!')

llm = LLM("<Your model here>")
llm.ask_with_history('What is the meaning of life?', fake_history)

The Chat History Class

Chat History handler. Keeps track of questions and answers as if they were a chat.

`add_question_answer(question, answer, clean=False)`

Adds a question and answer at once to the history.

Parameters:

Name	Type	Description	Default
`question`	`str`	The question	required
`answer`	`str`	The answer	required
`clean`	`bool`	Should the history be cleaned up after the cleanup threshold is reached?	`False`

`chat_as_llm(message, clean=False)`

Adds a message for the LLM (the one who has to handle the question)

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to add	required
`clean`	`bool`	Should the history be cleaned up after the cleanup threshold is reached?	`False`

`chat_as_user(message, clean=False)`

Adds a message for the user (the one who asks the question)

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to add	required
`clean`	`bool`	Should the history be cleaned up after the cleanup threshold is reached?	`False`