Weâve just released the Data lineage graph, a new feature that will let you visualize how the Data Sources, Pipes and Materialized Views on a workspace are connected. It will help you know where the data comes from, and where itâs going to, in a visual way.
There are two ways to use it:
- Clicking on the Data lineage graph button in the top of the left sidebar.
- Clicking on the Advanced information tab of a Data Source modal
We see some potential benefits for this feature:
- Better and simpler onboarding of new team members into big data projects.
- More visibility into which Data Sources are being unused
- More visibility of Pipes without an active endpoint
The Data Sources and Pipes will have different colors, to better help you understand whatâs going on. These are consistent across all the UI. Data Sourcesâ icons are purple if theyâre being used by any pipe, and red if theyâre not. Pipesâ icons are green if they expose an endpoint, or orange if the donât. And Materialized Views have green icons, with a âĄïž inside.
Weâre excited to see if this is a feature that makes sense to you, our users. Weâve built it to be able to solve the problems we found when dealing with large data projects. We already see a lot of avenues to improve it but weâd love to get your feedback.