We are introducing a completely new way of building your pipelines on 

 to shift our tool to focus on better supporting developers in the AI and data domain. The goal of VDP is to help you dynamically build versatile pipelines for complex and unique use cases that require minimal maintenance with high data observability.

The new pipeline builder UI allows you to edit YAML recipes next to the pipeline preview for improved debugging and data flow visualization. This means you can edit 

 files in your own editors like VSCode and simply import them into VDP, streamlining the recipe sharing process. Additionally, the editor offers smart hints and syntax highlighting, making it easier to build and modify recipes efficiently.

Watch this tutorial on how to build recipes in the new pipeline editor:

The pipeline preview offers a visual representation of data flow between components, striking a balance between code-level control and visual management. This feature is particularly beneficial for data engineers working on complex pipelines, as it provides real-time streaming feedback to simplify debugging. The preview also enhances component presentation, making it easier to build, test, and maintain sophisticated data pipelines efficiently.

 pipeline to transform a paragraph of unstructured text into structured JSON output in the new pipeline editor.

Learn more about this feature by joining our 

Instill AI changelog

Revamping the Pipeline Builder