Staff Data Platform Engineer

Reality Defender is a cybersecurity company that develops deepfake detection technology. The Staff Data Platform Engineer will own and scale the shared AI data platform, building reliable, reproducible data pipelines and infrastructure for research datasets and synthetic media generation.

Responsibilities:

• Own the end-to-end data platform for AI research: continuous ingestion, preprocessing, governance, and export of image, video, audio, and text data at high volume
• Complete and scale our Databricks lakehouse and distributed processing stack (Spark, Ray, or similar), making architecture and build-vs-buy decisions that hold up as data volume grows
• Build reusable infrastructure for in-house synthetic data generation, so researchers can produce new deepfake variants across modalities on demand instead of one-off
• Make every dataset reproducible and traceable through versioning, lineage, metadata, quality checks, and stable train/eval splits, so any model result can be tied back to the exact data behind it
• Own pipeline reliability, observability, and processing cost per dataset as core metrics of the platform
• Partner with ML researchers to understand their data needs and turn them into self-serve tools that speed up model iteration
• Mentor engineers and raise the bar on data engineering practices: code review, testing, documentation, and runbooks
• Work with Security and compliance stakeholders to make sure data handling, licensing, and access controls meet the standards our enterprise and government clients expect
• Help shape the long-term data platform roadmap with engineering and research leadership

Requirements:

• 8+ years building data platforms or data infrastructure, including staff- or lead-level ownership of production pipelines that generate, ingest, or deliver data at high volume
• Deep hands-on experience with distributed processing (Spark, Ray, or similar) and lakehouse platforms, ideally Databricks, on top of cloud object storage (S3, GCS, or Azure Blob)
• Production experience with Databricks, Unity Catalog, Delta Lake, and Spark on AWS, including performance tuning, access controls, and operational troubleshooting
• Experience working with large unstructured media such as video, audio, or images, including the realities of large files, decoding and transcoding, and metadata at scale
• A track record of building reproducible ML datasets: versioning, lineage, data quality validation, and orchestration (Airflow, Dagster, Databricks Workflows, or similar)
• Strong Python and SQL, and the judgment to own ambiguous platform problems end-to-end on a lean team
• Own infrastructure and deployment automation using Terraform and CI/CD, including AWS storage, IAM, compute, Databricks resources, and distributed processing environments
• Experience building synthetic data generation pipelines or running generative models (diffusion, voice cloning/TTS, video generation) at scale
• Background supporting ML research teams with training and evaluation data, especially for multimodal models
• Experience in security, fraud, trust & safety, or another adversarial domain, or at a Series A or B startup where scope regularly outpaces headcount