By Data Team Size
The goal of Metaplane is to help every company trust their data, from small but mighty teams of one, to large enterprises with complex data organizations. But while every data team faces challenges when ensuring trust, the specific ways that those challenges manifest generally differ based on the size of the team.
Startup (1-3 person data team)
Maybe your company just raised a Series A and is looking to Do Data Properly. Ad hoc HubSpot reports and munged CSVs are no longer cutting it for board meetings. As the first few members of your data team, there’s a lot of work ahead. Your mandate might be to stand up the first proper data stack at the company, while simultaneously delivering quick wins. Between negotiating the first vendor contracts and migrating data from production DB to your warehouse, you’re busy.
But with great responsibility comes great power. Now you have the opportunity to build a data culture of trust. Nimble data teams at startups like Gorgias, Appcues, and CreditGenie use Metaplane to:
- Move fast without breaking things through Data CI/CD. Using our Data CI/CD integration, you can understand the effect of code changes on downstream BI dependencies and your data itself within a PR. The best data issue is one that never happened in the first place.
- Be the first to know of data issues with Metaplane Monitors. As the first data team, there’s a lot of pressure on you. Maybe you feel like you can’t take days off, because if something goes wrong, it’s on your shoulders. With Metaplane monitors, you automated anomaly detection helping you continuously monitor for potential issues.
- Validate your the first migrations with Metaplane Monitors and Schema Change Detection. Using schema change detection, you can confirm that migrations are successfully completed with the shape and structure you expect. Using Metaplane monitors, you can confirm that the characteristics of your data (number of rows, distribution) are what you expect.
- Keep up with upstream prod migrations with Metaplane Schema Change Detection. Speed is the name of the game, and migrations in prod might inadvertently break your work. Find out about schema changes in your core transactional DBs instead of finding out when your dbt jobs break.
- Surface issues to relevant stakeholders directly. By integrating with communication channels like Slack, and routing based on data objects or tags, you're able to proactively alert the affected parties to kickstart building a data culture of trust.
- Save time. With implementations taking under 30 minutes, and no maintenance needed to update our automated machine-learning based monitors, once you set up Metaplane, the only time you need to come back is for insights to accelerate your data incident triaging.
Mid-market (3-10 person data team)
Did your company just celebrate a growth round with the goal of doubling or tripling revenue next year? Does it seem like new teammates are joining every week? Is your data team starting to specialize into different roles? As your company grows, the demands on data grows. Which is great. Use cases for data may be expanding beyond core decision support for executives into enhancing the product or powering other workflows. But data consumption is a double-edged sword.
Teams at this stage often stand at a precipice. It’s easy to get pulled into the wave of ad hoc requests, but it’s also an opportunity to maximize leverage. Teams at scaleups like Sigma, Drift, and Vendr use Metaplane to:
- Prevent data issues with Data CI/CD. As diligently checking the impact of every PR becomes unfeasible, let Metaplane’s Data CI/CD integration do it for you. Within PRs, you can see the effect of code changes on downstream BI dependencies and on data itself.
- Be the first to know of data issues with Metaplane Monitors. As the number of data sources and transformation expands, there’s many opportunities for breakage. Let Metaplane’s machine learning help identify anomalous changes (like a spike of null values or a change in the distribution of a column) or unexpected delays.
- Minimize alert fatigue with Metaplane Incidents. When data issues occur, it can be like a bowling ball hitting pins one after another. Instead of receiving a blast of alerts at once, let Metaplane determine which alerts are related based on lineage, timing, test type, and other relationships.
- Triage data issues with Impact Analysis. When issues occur, the natural follow-up question is: does this matter? Sometimes that question is hard to answer when there’s hundreds or thousands of dashboards and syncs. Instead of relying on your memory, Metaplane will automatically determine which assets are downstream of a data issue, so you can figure out if it’s worth your time and who to keep in the loop.
- Keep up with upstream data migrations with Schema Change Detection. The data team isn’t always kept appraised of updates in prod (e.g. new custom field in an upstream business application), which can sometimes cascade throughout the warehouse to break dashboards. Metaplane can automatically detect schema changes in databases and data warehouses, so you’re not caught by surprise again.
- Efficiently onboard teammates with End-to-End Lineage. Hiring and retaining great teammates is hard. Make sure their onboarding is as smooth as possible by giving them a full view of data assets within your infrastructure and help build their confidence through.
Enterprise (10+ person data team)
Your company might be gearing up to go public, or perhaps you’ve had a ticker since 2005. Maybe you’ve blitzed your way to scale, or maybe your data stack has been around for 20+ years, but in either situation: the impact of data is enormous. Your deployment of a new BI tool might impact hundreds or thousands of people. Migrations are heroic tasks.
Working at this calibre of company, the impact is immense, the stakes are high, but there are big opportunities to improve. Enterprise-calibre companies teams have a unique opportunity to manage complexity. Large teams at companies like Sothebys, SpotOn, and Weedmaps use Metaplane to:
- Foster a culture of trust in data by being the first to know of data issues. Let Metaplane be your Data Quality Engineer, or level up your colleagues responsible for data governance, with monitors that leverage automated machine learning that adapts to your data profile. And rest assured that alerts are highly configurable to your teams' preferences.
- Manage complexity through with End-to-End Lineage. The entire landscape of your data ecosystem might be too large to fit in one person’s head, but Metaplane can automatically extract lineage from ingestion to consumption without needing manual maintenance.
- Minimize data entropy through Usage Analytics. You might have thousands or tens of thousands of tables, dashboards, and syncs. But are they all being used? By parsing query history, Metaplane can help understand which data assets are regularly used, and therefore could use optimizations, and also those that could be cleaned up because they aren’t being used.
- Manage costs through Metaplane Spend Analysis. While we want to maximize the footprint of data within companies, we have to keep an eye on costs in order to optimize ROI. Metaplane can automatically monitor spend within your warehouse so you aren’t caught by surprise by queries that cost $50K because of unoptimized JOINs (we’ve seen this and worse).
- Ensure compliance with fine-grained monitors. Using Partition Tests and Custom SQL tests, your team can stay on top of data privacy and security concerns within the warehouse without having to spin up a sidecar process.
Updated over 1 year ago