Skip to main content

LLM Chat Complete

A system task to complete the chat query. It can be used to instruct the model's behavior accurately to prevent any deviation from the objective.

Definitions​

   {
     "name": "llm_chat_complete",
     "taskReferenceName": "llm_chat_complete_ref",
     "inputParameters": {
       "llmProvider": "openai",
       "model": "gpt-4",
       "instructions": "your-prompt-template",
       "messages": [
         {
           "role": "user",
           "message": "${workflow.input.text}"
         }
       ],
       "temperature": 0.1,
       "topP": 0.2,
       "maxTokens": 4,
       "stopWords": "and"
     },
     "type": "LLM_CHAT_COMPLETE"
   }

Input Parameters​

ParameterDescription
llmProviderChoose the required LLM provider. You can only choose providers to which you have access for at least one model from that provider.

Note:If you haven’t configured your AI / LLM provider on your Orkes console, navigate to the Integrations tab and configure your required provider. Refer to this doc on how to integrate the LLM providers with Orkes console and provide access to required groups.
modelChoose from the available language model for the chosen LLM provider. You can only choose models for which you have access.

For example, If your LLM provider is Azure Open AI & you’ve configured text-davinci-003 as the language model, you can choose it under this field.
instructionsSet the ground rule/instructions for the chat so the model responds to only specific queries and will not deviate from the objective.

Under this field, choose the AI prompt created. You can only use the prompts for which you have access.

Note:If you haven’t created an AI prompt for your language model, refer to this documentation on how to create AI Prompts in Orkes Conductor and provide access to required groups.
messagesChoose the role and messages to complete the chat query.

Role and messages in LLM Chat complete task

  • Under ‘Role,’ choose the required role for the chat completion. It can take values such as user, assistant, system, or human.
    • The roles “user” and “human” represent the user asking questions or initiating the conversation.
    • The roles “assistant” and “system” refer to the model responding to the user queries.
  • Under “Message”, choose the corresponding input to be provided. It can also be passed as variables.
temperatureA parameter to control the randomness of the model’s output. Higher temperatures, such as 1.0, make the output more random and creative. Whereas a lower value makes the output more deterministic and focused.

Example: If you're using a text blurb as input and want to categorize it based on its content type, opt for a lower temperature setting. Conversely, if you're providing text inputs and intend to generate content like emails or blogs, it's advisable to use a higher temperature setting.
stopWordsProvide the stop words to be omitted during the text generation process.

In LLM, stop words may be filtered out or given less importance during the text generation process to ensure that the generated text is coherent and contextually relevant.
topPAnother parameter to control the randomness of the model’s output. This parameter defines a probability threshold and then chooses tokens whose cumulative probability exceeds this threshold.

For example: Imagine you want to complete the sentence: “She walked into the room and saw a __.” Now, the top 4 words the LLM model would consider based on the highest probabilities would be:
  • Cat - 35%
  • Dog - 25%
  • Book - 15%
  • Chair - 10%
If you set the top-p parameter to 0.70, the AI will consider tokens until their cumulative probability reaches or exceeds 70%. Here's how it works:
  • Adding "Cat" (35%) to the cumulative probability.
  • Adding "Dog" (25%) to the cumulative probability, totaling 60%.
  • Adding "Book" (15%) to the cumulative probability, now at 75%.
At this point, the cumulative probability is 75%, exceeding the set top-p value of 70%. Therefore, the AI will randomly select one of the tokens from the list of "Cat," "Dog," and "Book" to complete the sentence because these tokens collectively account for approximately 75% of the likelihood.
maxTokensThe maximum number of tokens to be generated by the LLM and returned as part of the result. A token should be approximately 4 characters.

Output Parameters​

The task output displays the completed chat by the LLM.

Examples​



  1. Add task type LLM Chat Complete.
  2. Choose the LLM provider, model & prompt template.
  3. Provide the input parameters.

LLM Chat Complete Task