Data Engineer

SMART Compliance builds software that helps California cities and counties comply with environmental regulations by collecting, tracking, and reporting required data. The Data Engineer will own data pipelines, ensure data quality and auditability, support legacy migrations, apply AI tools to data challenges, and collaborate with technical teams, customers, jurisdictions, and hauler contacts.

Responsibilities:

• Build, maintain, and optimize ETL pipelines that ingest hauler, jurisdiction, and third-party data from spreadsheets, PDFs, APIs, and other sources
• Write per-source adapters and handle schema drift as customer data formats change from quarter to quarter
• Design and maintain PostgreSQL schemas, indexes, and JSONB models that keep the platform fast as our customer base grows
• Own data quality: build validation rules, reconciliation checks, and audit trails that tie every reported number back to its raw source record
• Support the migration of legacy customer data into our current platform and reconcile results between the two
• Apply AI tools to hard data problems such as document extraction, entity resolution, and mapping unfamiliar schemas, and know when the output needs a second look
• Investigate and explain data discrepancies to customers who are not technical, in writing and on calls
• Write tests, review code, and document what you build so the rest of the team can maintain it
• Contribute to our security and compliance practices, including controls around how customer data is stored, accessed, and processed

Requirements:

• At least 2 years of professional experience as a Data Engineer or in a closely related role
• Strong Python, SQL, and Linux skills. These are the three things we care about most
• Writing and optimizing advanced SQL queries
• Data modeling: ER diagrams, normalization, relational design
• Indexing and query performance tuning
• JSONB modeling in PostgreSQL
• Database security fundamentals
• Python for data work (pandas or Polars, SQLAlchemy)
• Tests, code review, and version control with git
• ETL construction and optimization
• Schema drift handling and per-source adapters
• Excel and CSV parsing at production quality, including merged cells, multi-row headers, and footnote rows
• PDF and document extraction
• Fuzzy matching and entity resolution
• JSON and XML processing
• Using and writing REST APIs
• Declarative validation (Pandera, Great Expectations, dbt tests, or similar)
• Reconciliation logic across sources and across system versions
• Auditability: knowing which raw record produced which number in a submitted report
• Explain technical problems clearly to people who are not technical, including city staff and hauler contacts
• Write documentation that other engineers can follow
• Comfortable working directly with customers when a data issue needs a conversation
• Location: Remote within the U.S
• Applicants must be authorized to work in the United States
• Familiarity with SB 1383, CalRecycle reporting, or solid waste and recycling data
• Experience with public-sector or government data
• ArcGIS, Survey123, or other geospatial data tools
• AWS infrastructure (EC2, RDS, S3, IAM)
• Docker and container-based deployment; Kubernetes
• Terraform or other infrastructure-as-code tools
• Multi-tenant and single-tenant database architecture
• Table partitioning strategies at scale
• MongoDB or other document stores
• API documentation with OpenAPI/Swagger
• Analytics and BI tools (Power BI, Tableau, or similar)
• Snowflake or other cloud data warehouses
• Ontologies, taxonomies, and knowledge graphs (RDF, SPARQL, SHACL)
• Security practices relevant to SOC 2 or similar frameworks