We are improving vendor connection resources by introducing Instill Integrations to easily setup and contain the connection details of any 3rd party vendors (i.e. OpenAI API key, Pinecone URL and API key, etc.) on Instill VDP. This will ensure higher observability by ensuring you see what vendor has been authorized for each integration and improve overall user experience when applied in the pipeline builder.

In contrast to Secret Management, this new integration feature will separate the component setup from the variable which means:

• Secrets will be used for avoiding repetition and will not be tied to any component type

• Integrations will be used to store connection setup details all in one place

To setup an integration, you can either do it through the Integrations page or within the pipeline builder.

Configuring integrations

The Integrations page will allow you to easily configure and protect your external connections all in one place, allowing you to quickly search & filter available integrations and configure new ones.

To setup your personal new integration, simply follow these steps:

1. Go to your Profile > Settings > Integrations

2. Click Connect on any available 3rd party vendor

3. Fill in the connection details

To setup your organizations new integration, simply follow these steps:

1. Go to your Profile > Your Organization > Organization > Integrations

2. Click Connect on any available 3rd party vendor

3. Fill in the connection details

Using integrations

When writing a pipeline recipe in the pipeline builder, you can reference the connection in the setup section within your component declaration:

component: 
  library-db:
    type: sql
    input:
      table-name: books
      update-data: ${variable.books}
    setup: ${connection.my-library-1}  # <-- reference connection 
    task: TASK_UPDATE

Next, we will also enable connecting with 3rd party services via OAuth.

Learn more about this feature by joining our Discord where we showcase these new features.