Role: Databricks Data Engineer/Architect
Location: Seattle, WA
Exp required : 12+ Years
Job Description:
- Design and develop scalable data pipelines using Databricks and Apache Spark.
- Build and maintain ETL/ELT processes for ingesting, transforming, and loading large volumes of data.
- Architect modern data platforms using the Databricks Lakehouse architecture.
- Implement data modeling solutions for data lakes, data warehouses, and analytics platforms.
- Optimize Spark jobs, clusters, and workloads for performance and cost efficiency.
- Integrate data from multiple sources, including cloud storage, databases, APIs, and streaming platforms.
- Work with cloud platforms such as AWS, Azure, or Google Cloud Platform, leveraging native data services.
- Establish data governance, security, data quality, and compliance standards.
- Develop and manage Delta Lake solutions, including data versioning and optimization.
- Collaborate with business stakeholders, data scientists, analysts, and engineering teams to define data requirements.
- Provide technical leadership, architecture guidance, and best practices for data engineering initiatives.
- Support CI/CD, automation, monitoring, and production deployment of data solutions.
Key Technical Skills Typically Required:
- Databricks
- Apache Spark (PySpark/Scala)
- SQL
- Delta Lake
- Data Lakehouse Architecture
- Azure Data Factory / AWS Glue (depending on cloud platform)
- Python
- Cloud Platforms (Azure, AWS, or Google Cloud Platform)
- ETL/ELT Development
- Data Modeling
- CI/CD and DevOps Practices
Ideal Candidate Profile:
- Strong hands-on experience with Databricks and Spark.
- Experience designing enterprise-scale data architectures.
- Expertise in cloud-based data engineering solutions.
- Ability to lead technical discussions and mentor engineering teams.
- Strong understanding of performance tuning, security, and data governance.