Key skills
ApacheAWSAzureCloudGraphQLKafkaRabbitMQSOAPSparkSQLTableauData EngineeringData LakeAnalyticsBIPower BISnowflakeDatabricksApache SparkS3IAMSQSGlueOAuthJWTCI/CD
About this role
Role Overview
- Design, build, and maintain cloud-native data pipelines and data products across Azure and AWS using Databricks and Snowflake.
- Lead and contribute to the modernization and migration of on‑prem and legacy data platforms to cloud-based solutions.
- Implement batch and streaming data processing patterns using Spark and cloud-native services.
- Partner with data governance, security, and risk teams to ensure data products comply with enterprise governance, data privacy, and regulatory requirements.
- Enable secure data sharing and access patterns across domains and platforms using appropriate controls.
- Define and promote data engineering best practices, including CI/CD, testing, observability, performance tuning, and cost optimization.
- Collaborate with product owners and analytics teams to translate business requirements into well-modeled, high-quality datasets.
- Work closely with cloud and security architects to implement secure, scalable, and resilient data solutions.
- Support and mentor junior engineers through design reviews, code reviews, and technical guidance.
Requirements
- Bachelor’s degree, or equivalent work experience
- Six to eight years of relevant experience
- Experience with data architecture and platform design in large enterprises.
- Knowledge of data sharing, data mesh, or domain-driven data architecture concepts.
- Strong problem-solving skills and a track record of delivering scalable, efficient data solutions.
- Strong hands-on experience with Azure Data Platform services, including: Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics (or Fabric equivalent experience)
- Experience with AWS data services, such as AWS Glue, S3, and event-driven integrations.
- Deep experience with Databricks (Spark, Delta Lake, performance tuning).
- Strong working knowledge of Snowflake, including data modeling, ingestion patterns (e.g., Snowpipe), and data sharing.
- Expertise in Apache Spark for large-scale data processing.
- Experience building batch and near-real-time data pipelines.
- Strong SQL skills and experience with dimensional and analytical data modeling.
- Experience designing reusable, domain-oriented data products.
- Experience with API-based integrations (REST; familiarity with SOAP and GraphQL is a plus).
- Hands-on experience integrating with API gateways.
- Understanding of messaging and streaming platforms such as Kafka, MQ, AWS SQS, or RabbitMQ.
- Strong understanding of IAM, RBAC, OAuth 2.0, TLS/mTLS, and JWT.
- Experience implementing secure data access patterns in cloud environments.
- Familiarity with data cataloging, lineage, and metadata management concepts.
- Experience enabling self-service analytics and BI using tools such as Power BI, Tableau, or equivalent.
Tech Stack
- Apache
- AWS
- Azure
- Cloud
- GraphQL
- Kafka
- RabbitMQ
- SOAP
- Spark
- SQL
- Tableau
Benefits
- Healthcare (medical, dental, vision)
- Basic term and optional term life insurance
- Short-term and long-term disability
- Pregnancy disability and parental leave
- 401(k) and employer-funded retirement plan
- Paid vacation (from two to five weeks depending on salary grade and tenure)
- Up to 11 paid holiday opportunities
- Adoption assistance
- Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year unless otherwise provided by law