Senior Data Engineer

Lumin Digital is hiring a Senior Data Engineer to design, build, and optimize the foundational data infrastructure and tools supporting its data ecosystem, with a primary focus on Databricks. The role develops scalable data platform solutions, integrations, and reusable tools while improving performance, reliability, security, and developer productivity.

Responsibilities:

• Design, develop, and maintain scalable, reliable, and cost-effective data platform architecture and foundational capabilities, with a primary focus on Databricks
• Build and optimize data pipelines, integrations, and reusable tools that streamline data processing, improve developer productivity, and accelerate delivery across engineering and analytics teams
• Monitor and optimize data platform performance, reliability, and resource utilization, identifying opportunities to improve efficiency and reduce operational costs
• Collaborate with security and engineering teams to ensure data platform solutions align with established security standards, data governance practices, and applicable regulatory and compliance requirements
• Partner with cross-functional stakeholders to identify shared platform needs, prioritize foundational improvements, and contribute to technical planning and architectural decisions
• Perform other duties as assigned

Requirements:

• Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
• 5+ years of experience in software engineering, data engineering, or related technical disciplines
• 5+ years of hands-on Databricks experience designing, developing, administering, and maintaining scalable cloud-based data platforms
• Experience building data pipelines, integrations, automation, and reusable tools that support data processing and improve engineering productivity
• Experience configuring, monitoring, troubleshooting, and optimizing data platform infrastructure for performance, reliability, scalability, and cost efficiency
• Experience working with cloud infrastructure and services, preferably AWS, with experience in Azure or Google Cloud also valued
• Experience developing and maintaining solutions using Scala, Python, SQL, Apache Spark, or similar data engineering technologies
• Experience implementing automated infrastructure provisioning, configuration management, and deployment processes using infrastructure-as-code and CI/CD practices
• Experience supporting production environments, resolving complex technical issues, and implementing improvements that increase platform stability and operational efficiency
• Experience collaborating with cross-functional engineering teams to deliver shared technical capabilities and platform improvements
• Demonstrates humility, integrity, accountability, and a strong sense of ownership. Willing to collaborate across teams, take a hands-on approach to solving challenges, and contribute wherever needed to achieve shared outcomes
• Commitment to continually improving yourself
• Strong communication and collaboration skills, with the ability to document technical solutions, share knowledge, and contribute to engineering standards and best practices
• Strong understanding of modern data platform architecture, distributed data processing, and cloud-based infrastructure
• Knowledge of Databricks platform capabilities, including workspace administration, compute configuration, workflows, access management, monitoring, and cost optimization
• Strong programming and scripting skills, particularly in Python and SQL, with the ability to develop reliable, maintainable, and reusable solutions
• Understanding of Apache Spark, Delta Lake, and data processing and storage optimization techniques
• Knowledge of cloud infrastructure, security, networking, and resource management, particularly within AWS environments
• Ability to design and implement platform integrations, automation, and foundational tools that simplify development and accelerate delivery for other engineering teams
• Understanding of infrastructure-as-code, CI/CD, and automated deployment practices
• Knowledge of data governance, security best practices, access controls, and platform reliability principles
• Familiarity with streaming data platforms and event-driven architectures, such as Apache Kafka or Confluent, is preferred