Key skills
PythonpandasPySparkDatabricksApache SparkAWS S3Cloud-based data architecturesData pipeline testingVersion controlCI/CD practicesSQLNoSQL databasesGeospatial data formatsGeoJSONShapefilesPostGISWorkflow orchestration toolsDockerKubernetesMachine learning pipelinesNoSQLAIMachine LearningPandasETLELTData EngineeringAnalyticsSparkAWSS3Version ControlCI/CD
About this role
Urban SDK is shaping the Future of Smart Cities through geospatial AI technology. They are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines that support geospatial traffic analytics applications.
Responsibilities:
- Design, implement, and maintain scalable data pipelines and ETL/ELT workflows on Databricks and cloud platforms
- Manage large-scale geospatial and temporal datasets stored in AWS S3
- Collaborate with data scientists to productionize machine learning models and ensure smooth data availability
- Implement data validation, testing, and monitoring frameworks to ensure data accuracy, consistency, and reliability
- Optimize data storage and processing strategies to handle high volumes of traffic and mobility data efficiently
- Develop and maintain documentation for data workflows, architecture, and processes
- Work closely with cross-functional teams to understand data requirements and ensure timely delivery
- Stay up-to-date with the latest trends and best practices in data engineering, cloud technologies, and big data processing
Requirements:
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field
- 3+ years of experience as a data engineer or in a similar role
- Strong proficiency in Python and associated libraries for data engineering (pandas, PySpark, etc.)
- Hands-on experience with Databricks and Spark for large-scale data processing
- Experience with AWS services, especially S3, and knowledge of cloud-based data architectures
- Solid understanding of data pipeline testing, version control, and CI/CD practices
- Experience with SQL and NoSQL databases
- Strong problem-solving skills and attention to detail
- Familiarity with geospatial data formats and processing (GeoJSON, Shapefiles, PostGIS)
- Experience with workflow orchestration tools (Databricks, Prefect, or similar)
- Knowledge of containerization (Docker/Kubernetes) and cloud-native data solutions
- Experience supporting machine learning pipelines in production