Using AI Models or LLMs

In this guide, you will learn how to leverage the power of Orkes Conductor and AI models, such as large language models (LLMs) or embedding models, in your applications. Here is an overview of using AI models in Orkes Conductor to build an AI-powered application:

1. Choose an AI model for your use case.
2. Integrate your chosen AI model with your Orkes Conductor cluster.
3. (If required) Create an AI prompt using Orkes’ AI Prompt Studio, where you can tune the prompt and test it with different models.
4. (If required) Integrate your chosen vector database with your Orkes Conductor cluster.
5. Set access limits to the AI model, prompt, or vector database to govern which applications or groups can use them.
6. Add a system AI task to your workflow and configure it for the chosen model and prompt template.

Step 1: Choose an AI model

The AI models offered by different providers have a wide range of applications, such as:

• Text generation and completion
• Text summarization
• Language translation
• Sentiment analysis
• Text classification
• Embedding generation

Not all models are suitable for all these use cases, as some models are better at specific tasks due to their architecture and training. For example, OpenAI offers models like "text-embedding-ada-002" designed specifically to generate embeddings.

Available model providers

The following model providers are available for integration with Orkes Conductor:

• Azure + OpenAI
• OpenAI
• Cohere​​
• Google Vertex AI
• Google Gemini AI
• Anthropic Claude
• Hugging Face
• AWS Bedrock Anthropic
• AWS Bedrock Cohere
• AWS Bedrock Llama2
• AWS Bedrock Titan
• Mistral

Review the model provider’s official documentation to determine which models suit your use case.

Step 2: Integrate an AI model

Before using an AI model in a workflow, you must integrate it with your Orkes Conductor cluster.

To integrate an LLM:

1. Go to Integrations from the left menu on your Orkes Conductor cluster.

2. Select + New integration from the top-right of your window.

3. In the AI/LLM section, select + Add to integrate your preferred model provider.

   

4. Enter the required parameters for the chosen provider.

   note

   The integration configuration differs with each model provider. For detailed steps on integrating with each provider, refer to AI/LLM Integrations.

5. (Optional) Toggle the Active button off if you don’t want to activate the integration instantly.

6. Select Save.

Add the preferred models

Once the AI/LLM integration is added, you can begin adding models from the provider.

List of models

• Azure OpenAI models
• OpenAI models
• Cohere models
• Google Vertex AI models
• Google Gemini AI models
• Anthropic Claude models
• Hugging Face models
• AWS Bedrock Anthropic models
• AWS Bedrock Cohere models
• AWS Bedrock Llama 2 models
• AWS Bedrock Titan models
• Mistral models

To add a model:

1. In Integrations, select the + icon next to your newly-created integration.
2. Select + New model.
3. Enter the model name and an optional description for the model.
4. (Optional) Toggle the Active button off if you don’t want to activate the model instantly.
5. Select Save.

Step 3: Create an AI prompt

If you are using an LLM Text Complete or LLM Chat Complete task, you must use a prompt to interact with it. You can create a prompt in the AI Prompt Studio to be used with system AI tasks in a workflow. Refer to Using AI Prompts for more information.

Step 4: Integrate a vector database

You must also integrate a vector database if you are using one of the following system AI tasks:

• LLM Store Embeddings
• LLM Index Text
• LLM Index Document
• LLM Get Embeddings
• LLM Search Index

Refer to Using Vector Databases for more information.

Step 5: Set access limits

As best practice, use Orkes’ RBAC feature to govern which applications or user groups can access the AI models, prompts, and vector databases.

To provide access to an application or group:

1. Go to Access Control > Applications or Groups from the left menu on your Orkes Conductor cluster.

2. Create a new group/application or select an existing one.

3. In the Permissions section, select + Add Permission.

4. In the Integration tab, select the required AI models and/or vector databases and toggle the necessary permissions.

   

5. In the Prompt tab, select the required prompt and toggle the necessary permissions.

6. Select Add Permissions.

The group or application can now access the AI resources according to the configured permissions.

Step 6: Add a system AI task to your workflow

With the AI/LLM integration and AI prompt ready, you can add a system AI task to your workflow and configure the task for the chosen model and prompt.

To add a system AI task:

1. Go to Definitions > Workflow from the left menu on your Orkes Conductor cluster.

2. Create a new workflow or select an existing workflow.

3. Add an AI task and select the desired model provider and model.

4. Configure the remaining task parameters. If you are using an LLM Text Complete or LLM Chat Complete task, you can select the prompt template from the dropdown based on the chosen model.

   note

   Refer to the AI Task Reference for more details on configuring the task parameters.

5. Save the workflow.

Once the workflow is saved, you can test the newly added AI task or run the workflow.

Examples

AI-powered translator

Check out the tutorial on how to build an AI-powered language translator.

More resources

• Using AI Prompts
• Using Vector Databases
• AI/LLM Integration Guides
• AI Task Reference