Senior Software Engineer

LanceDB is an AI-native multimodal lakehouse company serving data-intensive AI applications. The Senior Software Engineer will expand Lance and LanceDB within the data infrastructure ecosystem by improving scale and performance, building integrations with data and AI systems, simplifying distributed operations, and enhancing usability and maintainability.

Responsibilities:

• Designing and maintaining efficient distributed Lance dataset operations
• Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino
• Working on table formats, data encodings, and various aspects of the Lance format in Rust
• Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems
• Operating and improving internal data processing infrastructure
• Promoting the Lance format in open-source communities and at Big Data conferences

Requirements:

• 10+ years of experience building high-performance databases, big data systems, or large-scale data services
• Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)
• Strong experience with high-performance computing in C++, Java, and/or Scala
• Experience with Rust (or willingness to learn it)
• Proven ability to move fast, work independently, and collaborate with a high-caliber team
• Contributor, committer, or PMC member in Apache or other large open-source projects
• Experience with Apache Arrow, DataFusion, Parquet, Iceberg, or Delta Lake
• Track record of driving large features or integrations in distributed systems
• Strong community presence and passion for open-source collaboration