How Roblox Reduces Spark Join Query Costs With Machine Learning

Abstract Every day on Roblox, 70 million users engage with millions of experiences, totaling 16 billion hours quarterly. This interaction generates a petabyte-scale data lake, which is enriched for analytics and machine learning (ML) purposes. It’s resource-intensive to join fact and dimension tables in our data lake, so to optimize this and reduce data shuffling, […]

How Roblox Reduces Spark Join Query Costs With Machine Learning Optimized Bloom Filters - Roblox Blog

PDF) The Optimization of Cost-Model for Join Operator on Spark SQL Platform

Orchestrating Data Pipelines at Lyft: Comparing Flyte and Airflow

What I Learned From Attending TWIMLCon 2021, by James Le

Roblox on LinkedIn: How Roblox Reduces Spark Join Query Costs With Machine Learning Optimized…

Advanced Spark SQL Joins: an Optimization Technique - Knoldus Blogs

Advanced Spark SQL Joins: an Optimization Technique - Knoldus Blogs

Machine Learning with XGBoost on Qubole Spark Cluster

How Databricks Optimizes the Spark SQL Joins, by dezimaldata

ROBLOXRSS (@ROBLOXRSS) / X

Making Sense of the Metadata: Clustering 4,000 Stack Overflow tags with BigQuery k-means - Stack Overflow

The art of joining in Spark. Practical tips to speedup joins in…, by Andrea Ialenti