Raval West is seeking a Data Platform Engineer to build scalable data pipelines and transformations in Databricks, focusing on geospatial data processing. The engineer will integrate ArcGIS and AWS data sources, develop Python and SQL transformations, and support secure, governed data access through Unity Catalog.

Responsibilities:

Build and maintain Databricks data pipelines and transformations
Ingest and process spatial data using the ArcGIS Python API
Develop scalable data transformations using Python, PySpark, and SQL
Integrate Databricks with AWS S3 data sources
Create and maintain Delta tables and curated datasets
Configure data access and governance using Unity Catalog
Implement data-quality checks, monitoring, and performance optimization

Requirements:

Strong hands-on experience with Databricks, Python, and SQL
Experience building production data pipelines and transformations
Experience with spatial data and the ArcGIS Python API
Knowledge of AWS S3 and cloud-based data architectures
Experience with Delta Lake and Unity Catalog
Understanding of spatial formats, coordinate systems, and geospatial transformations

Data Platform Engineer - Databricks

Key skills

About this role

Responsibilities:

Requirements: