We’ve just released the Data lineage graph, a new feature that will let you visualize how the Data Sources, Pipes and Materialized Views on a workspace are connected. It will help you know where the data comes from, and where it’s going to, in a visual way.
There are two ways to use it:
- Clicking on the Data lineage graph button in the top of the left sidebar.
- Clicking on the Advanced information tab of a Data Source modal
We see some potential benefits for this feature:
- Better and simpler onboarding of new team members into big data projects.
- More visibility into which Data Sources are being unused
- More visibility of Pipes without an active endpoint
The Data Sources and Pipes will have different colors, to better help you understand what’s going on. These are consistent across all the UI. Data Sources’ icons are purple if they’re being used by any pipe, and red if they’re not. Pipes’ icons are green if they expose an endpoint, or orange if the don’t. And Materialized Views have green icons, with a ⚡️ inside.
We’re excited to see if this is a feature that makes sense to you, our users. We’ve built it to be able to solve the problems we found when dealing with large data projects. We already see a lot of avenues to improve it but we’d love to get your feedback.
