Monitor your organization in Tinybird¶
The Tinybird organizations feature is only available to Enterprise or Dedicated plan customers (see "Tinybird plans").
Use the organizations section to monitor their consolidated Tinybird usage in one place. The section provides a single dashboard for the overview of all Workspaces within an organization, and unlocks specific Workspace-level actions. It includes dashboards for monitoring consumption of storage and processed data, Workspaces, and organization members. Usage data is also available through an HTTP API.
The organizations section consists of the following areas:
- Overview
- Workspaces
- Members
- Monitoring
Access the organizations section¶
To access the organizations UI, log in as an administrator and select your Workspace name, then select your organization name.
To add another user as an organization administrator, follow these steps:
- Navigate to the Your organization page.
- Go to the Members section.
- Locate the user you want to make an administrator.
- Select Organization Admin next to their name.
This grants administrator access to the selected users.
Usage overview¶
The Usage page shows details about your platform usage against your billing plan commitment followed by a detailed breakdown of your consumption. Only billable Workspaces are in this view. Find non-billable Workspaces in the Workspaces tab.
Processed data¶
The first metric shows an aggregated summary of your processed data. This is aggregated across all billable Workspaces included in your plan. Processed data is cumulative over the plan's billing period.
Storage¶
The second metric shows an aggregated summary of your current storage. This is aggregated across all billable Workspaces included in your plan. Storage is the maximum storage used in the past day.
Contract¶
The third metric shows the details of your current contract, including the plan type and start/end dates of your plan period. If your plan includes usage limits, for example commitments on an Enterprise plan, your commitment details are also shown here.
For both the Processed data and Storage metrics, the summary covers the current billing period. For Enterprise plans this covers the term of your current contract. For monthly plans, it's the current month.
After the summary, the page shows a breakdown of Processed data and Storage per Workspace and Data Source. The calculation of these metrics is the same as previously explained for the summary section, but on an individual basis.
Organization Service Data Sources¶
The Charts in the Overview page get their data from Organization Service Data Sources. The complete list of available ones is:
organization.workspaces: lists all Organization Workspaces and related information (name, IDs, databases, plan, when it was created, and whether it has been soft-deleted).organization.processed_data: information related to all processed data per day per workspace.organization.datasources_storage: equivalent to tinybird.datasources_storage but with data for all Organization Workspaces.organization.pipe_stats: equivalent to tinybird.pipe_stats but with data for all Organization Workspaces.organization.pipe_stats_rt: equivalent to tinybird.pipe_stats_rt but with data for all Organization Workspaces.organization.datasources_ops_log: equivalent to tinybird.datasources_ops_log but with data for all Organization Workspaces.organization.data_transfer: equivalent to tinybird.data_transfer but with data for all Organization Workspaces.organization.jobs_log: equivalent to tinybird.jobs_log but with data for all Organization Workspaces.organization.sinks_ops_log: equivalent to tinybird.sinks_ops_log but with data for all Organization Workspaces.organization.bi_stats: equivalent to tinybird.bi_stats but with data for all Organization Workspaces.organization.bi_stats_rt: equivalent to tinybird.bi_stats_rt but with data for all Organization Workspaces.
Only Organization Admins are able to run these queries. To query these Organization Service Data Sources, go to any Workspace that belongs to the Organization, and use these as you would a regular Service Data Source from the Playground or within Pipes. Use the admin user@domain Token of an Organization Admin. You can also copy your user admin Token and make queries using your preferred method, like tb sql.
Workspaces¶
This page displays details of all your Workspaces, their consumption, and whether they're billable or not. Using the date range selector at the top of the page, you can adjust the time of the data displayed in the table.
The table shows the following information:
- Workspace name
- Processed data: Processed data is cumulative over the selected time range.
- Storage: Storage is the maximum storage used on the last day of the selected time range.
- Plan type: Billable or free. Usage in a billable Workspace counts towards your billing plan.
New Workspaces that are created by a user with an email domain linked to (or matching) an Organization are automatically added to that Organization. The new Workspace then automatically shows up here in your Organization's Consumption metrics and listed Workspaces. If you encounter any challenges with creating a new Workspace in your Organization, contact support@tinybird.co.
To delete Workspace, select the checkbox of a Workspace name, followed by the Delete button. You don't need to be a user in that Workspace to delete it.
Members¶
Members shows details of your Organization members, the Workspaces they belong to, and their roles.
User roles:
- Admins can do everything in the Workspace.
- Guests can do most things, but they can't delete Workspaces, invite or remove users, or share Data Sources across Workspaces.
- Viewers can't edit anything in the main Workspace Branch, but they can use Playgrounds to query the data, as well as create or edit Branches.
The table shows the following information:
- Workspaces and roles
To view the detail of a member’s Workspaces and roles, select the arrow next to the Workspace count. A menu shows all the Workspaces that user is part of, plus their role in each Workspace.
To change a user’s role or remove them from a Workspace, hover over the Workspace name and follow the arrow. Select a new role from Admin, Guest, or Viewer, or remove them from the Workspace. You don't need to be a user in that Workspace to make changes to its users. As mentioned, you can also make a user an organization admin from this page.
To remove a user from the organization, select Remove member in the menu.
You can see if there are Workspaces where that user is the only admin and if the Token associated to the email has had activity in the last 7 days.
Monitoring endpoints¶
To monitor the usage of your Organization use the Organization Service Data Sources.
The endpoints page shows details about the APIs that allow you to export, or integrate external tools with, your usage data.
There are two APIs available:
Processed data¶
The first API shows a daily aggregated summary of your processed data per Workspaces.
Storage¶
The last API shows a daily aggregated summary of your current storage per Workspaces. You can select the time by editing the parameters in the URL.
Processed data
| Field | Type | Description |
|---|---|---|
| day | DateTime | Day of the record |
| workspace_id | String | ID of the Workspace. |
| read_bytes | UInt64 | Bytes read in the Workspace that day |
| written_bytes | UInt64 | Bytes written in the Workspace that day |
Storage
| Field | Type | Description |
|---|---|---|
| day | DateTime | Day of the record |
| workspace_id | String | ID of the Workspace. |
| bytes | UInt64 | Maximum Bytes stored in the Workspace that day |
| bytes_quarantine | UInt64 | Maximum Bytes stored in the Workspace quarantine that day |
Dedicated infrastructure monitoring¶
The following features are in public beta and may change without notice. If you have feedback or suggestions, contact Tinybird at support@tinybird.co or in the Community Slack.
If your organization is on an infrastructure commitment plan, Tinybird offers two ways of monitoring the state of your dedicated clusters: using the organization.metrics_logs service Data Source, or through the Prometheus endpoint /v0/metrics, which you can integrate with the observability platform of your choice.
Billing dashboard¶
You can track your credits usage from the Billing section under Your organization. The dashboard shows your cumulative credits usage and estimated trend against the total, and warns you if you're about to run out of credits. For more details, you can access your customer portal using the direct link.
Cluster load chart¶
You can check the current load of your clusters using the chart under Your organization, Usage. Select a cluster in the menu to see all its hosts, then select a time. Each line represents the CPU usage of the host. When you select a host, the dotted line represents the total amount of CPUs available to the cluster.
metrics_logs Service Data Source¶
The metrics_logs Service Data Source is available in all the organization's workspaces. As with the rest of Organization Service Data Sources, it's only available to Organization administrators. New records for each of the metrics monitored are added every minute with the following schema:
| Field | Type | Description |
|---|---|---|
| timestamp | DateTime | Timestamp of the metric |
| cluster | LowCardinality(String) | Name of the cluster |
| host | LowCardinality(String) | Name of the host |
| metric | LowCardinality(String) | Name of the metric |
| value | String | Value of the metric |
| description | LowCardinality(String) | Description of the metric |
| organization_id | String | ID of your organization |
The available metrics are the following:
| Metric | Description |
|---|---|
| MemoryTracking | Total amount of memory, in bytes, allocated by the server. |
| OSMemoryTotal | The total amount of memory on the host system, in bytes. |
| InstanceType | Instance type of the host . |
| Query | Number of executing queries. |
| NumberCPU | Number of CPUs. |
| LoadAverage1 | The whole system load, averaged with exponential smoothing over 1 minute. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO. |
| LoadAverage15 | The whole system load, averaged with exponential smoothing over 15 minutes. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO. |
| CPUUsage | The ratio of time the CPU core was running OS kernel (system) code or userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). |
Prometheus metrics endpoint¶
The Prometheus endpoint is available at /v0/metrics. Use your organization's observability Token, which you can find in the Prometheus tab under Monitoring, Your organization.
Tinybird reports all the metrics available in the service Data Source, except for InstanceType, which is reported as a metric label.
Refreshing your organization's Observability Token¶
If your organization's observability Token gets compromised or is lost, refresh it using the following endpoint:
/v0/organizations/<organization-id>/tokens/Observability%20%28builtin%29/refresh?token=<your-user-token>
You must use your user token for this call, which you can copy from any of your workspaces.