Google Cloud Storage Integration with Orkes Conductor

Available since

• v5.3.0 and later

Integrating Google Cloud Storage with Orkes Conductor lets you build workflows that interact with your Google Cloud Storage using the following operations:

• Upload Object From Base64
• Download Object Base64
• List Objects
• Update Object
• Delete Object
• List Buckets
• Create Bucket
• Get Bucket
• Update Bucket
• Delete Bucket

This guide explains how to set up and use the integration. Here's an overview:

1. Get the required credentials from Google Cloud Console.
2. Configure a new Google Cloud Storage integration in Orkes Conductor.
3. Use Google Cloud Storage integration in workflows.

Step 1: Get the Google Cloud Console credentials

To integrate Google Cloud Storage with Orkes Conductor, you need a Google Cloud service account with the appropriate Cloud Storage permissions and its JSON key file.

To create a service account and generate a key:

1. Go to the Google Cloud Console and select your project.

2. Navigate to IAM & Admin > Service Accounts.

3. Select + Create service account, enter a name and description, and select Create and continue.

4. Assign one of the following IAM roles based on the access level required:

   • Storage Object Viewer: Read-only access to objects.
   • Storage Object User: Read and write access to objects.
   • Storage Admin: Full control over buckets and objects.

   Refer to the following table for the minimum IAM role required for each operation:

   Operation	Minimum IAM Role
   Upload Object From Base64	Storage Object Creator
   Download Object Base64	Storage Object Viewer
   List Objects	Storage Object Viewer
   Update Object	Storage Object Admin
   Delete Object	Storage Object Admin
   Get Bucket	Storage Object Viewer
   Update Bucket	Storage Admin
   Delete Bucket	Storage Admin
   List Buckets	Storage Admin
   Create Bucket	Storage Admin

   note

   The List Buckets and Create Bucket operations require project-level IAM permissions. If your service account only has bucket-level permissions, these operations will fail with a 403 error.

5. Select Continue, then Done.

6. Click on the newly created service account, go to the Keys tab, and select Add Key > Create new key.

7. Choose JSON as the key type and select Create. The key file downloads automatically.

Keep the downloaded JSON file ready; you'll need its full contents in the next step.

Step 2: Add an integration for Google Cloud Storage

After obtaining the credentials, add a Google Cloud Storage integration to your Conductor cluster.

To create an integration:

1. Go to Integrations > Connected Apps from the left navigation menu on your Conductor cluster.
2. Select + New integration.
3. In the Cloud section, choose Google Cloud Storage.
4. Select + Add and enter the following parameters:

Parameter	Description	Required/Optional
Integration name	A name for the integration.	Required.
Scope	The OAuth2 scope for Cloud Storage API access. Supported values: Read Only:Grants read-only access to buckets and objects. Read & Write: Grants read and write access to buckets and objects. Full Control: Grants full control over buckets and objects, including updating and deleting.	Required.
Service Account JSON	Paste the full contents of the Google Cloud service account key JSON file downloaded in Step 1.	Required.
Default Bucket Name	The default Google Cloud Storage bucket to use when a bucket is not explicitly provided in a workflow task.	Optional.
Description	A description of the integration.	Required.

5. (Optional) Toggle the Active button off if you don't want to activate the integration instantly.
6. Select Save.

Step 3: Use Google Cloud Storage integration in workflows

Once the integration is ready, this can be used directly within the workflows.

To use Google Cloud Storage integration in a workflow:

1. Go to Definitions > Workflow from the left navigation menu on your Conductor cluster.
2. Select + Define workflow.
3. In your workflow, select the (+) icon and select Connected Apps.

4. In Add Task panel on the right, search for Google Cloud Storage, and select the integration created in Step 2.

The following operations are available for use with this integration.

Operation	Description	Supported Scopes
Upload Object From Base64	Upload a file to a Google Cloud Storage bucket using base64-encoded content.	Full Control Read & Write
Download Object Base64	Retrieve an object's content from a Google Cloud Storage bucket as base64-encoded data.	Full Control Read & Write Read Only
List Objects	List objects in a Google Cloud Storage bucket, with optional prefix filtering.	Full Control Read & Write Read Only
Update Object	Replace an object's content or update its metadata in Google Cloud Storage.	Full Control
Delete Object	Permanently remove an object from a Google Cloud Storage bucket.	Full Control Read & Write
List Buckets	List all Google Cloud Storage buckets in the project.	Full Control Read & Write Read Only
Create Bucket	Provision a new Google Cloud Storage bucket.	Full Control Read & Write
Get Bucket	Retrieve metadata for a specific Google Cloud Storage bucket, including location, storage class, and versioning settings.	Full Control Read & Write Read Only
Update Bucket	Modify a Google Cloud Storage bucket's settings, such as versioning or storage class.	Full Control
Delete Bucket	Remove a Google Cloud Storage bucket.	Full Control Read & Write

5. Select the required operation, configure the parameters, and select Save > Confirm.
6. Select Execute to run the workflow.

For the complete operations parameters and output reference, see Google Cloud Storage Operations Reference.