DataStax Builds Trust in Product Usage Data with Telmai
Fully automated data observability empowers trust in product usage data across 36,000 clusters. DataStax is a real-time data company. DataStax helps enterprises mobilize real-time data and quickly build the smart, high-scale applications required to become data-driven businesses.
As a result, product operations, reliability, performance, and availability are the highest priority for DataStax. A leading indicator of their product health is strong and growing product adoption and usage. To monitor and report on usage, DataStax initially built a homegrown solution.
Solution: ML-based data observability automates data quality for usage reporting across 36,000 clusters
With the realization and learnings from the past, and to automate data quality at scale, the team decided to invest in the ML-based data observability solution of Telmai. With this automation, the high-caliber data engineering team could put their focus and resources on their core product advancement and leave the observability and monitoring to Telmai.
Today Telmai is used to monitor the actual data values, drifts, and anomalies in DataStax’s product usage. With Telmai, DataStax tracks
Data accuracy, completeness, and uniqueness
Drifts and trends in data over time (e.g., monitoring usage growth)
Telmai is placed between the data coming from the raw store (log data) and Google BigQuery. Selected tables and anonymized attributes from BigQuery are loaded into Telmai for tracking and monitoring. With Telmai Data Observability, DataStax is able to:
Track users, clusters, and organizations
Monitor the number of new clusters and conversion date (from sign-up) on a daily basis
Observe drifts in volume/record count of clusters and investigate those records using a visual, no code data investigator product
Track total read and total write within a cluster, segmented by usage date
Detect usage drifts on total read and total write compared to the predicted thresholds
Identify clusters with no usage
Today Telmai is deployed to track and monitor over 36,000 clusters, with an average of 10,000 daily active clusters.
“To prepare data for product usage analysis we needed data quality metrics beyond monitoring operational data pipelines and job status checks. While we continue to monitor the quality of the pipeline, we chose Telmai to detect the quality of the data that moves through the pipeline.”
Data & Analytics Leader, DataStax
“With Telmai we no longer have to think of all possible data issues and can leave anomy detection and unknown outliers to Temai`s ML algorithms to catch. This helps us prevent unexpected data quality issues and refocus our engineering efforts on advancing our products.”
Data Science & Analytics, DataStax
Start your data observability today
Get a demo from a solutions engineer in less than 24 hours