Software Engineer 3
San Diego, California, United States of America
Contract
2 hours ago
Key skills
PythonSQLRAIMLLLMOpenAIClaudeAnthropicRAGData EngineeringAnalyticsDatabricksSparkAzureAzure DevOpsSalesforceSAP
About this role
Technical Skills Must Have
- Databricks Mosaic AI
- Databricks SQL
- Databricks Unity Catalog
- GraphRag
- SalesForce
- SAP
Job Description
- Responsible for data engineering and AI agent development
Responsibilities
- Design, build, and optimize medallion architecture pipelines (Bronze Silver Gold) in Databricks using Delta Lake
- Ingest and process unstructured data (PDFs, images, documents, logs) from enterprise source systems (SAP, Salesforce, TrackWise, Azure DevOps, etc.)
- Curate and model data into structured, query-ready schemas to power AI agents and analytics applications
- Develop and maintain data quality frameworks, validation checks, and monitoring across pipeline stages
- Collaborate with AI/ML engineers to prepare datasets for LLM-powered agents (Claude, GPT) including embeddings, chunking strategies, and retrieval-augmented generation (RAG) pipelines
- Support Unity Catalog governance, access controls, and schema management across dev/prod workspaces
- Partner with cross-functional teams (Quality, Regulatory, IT, R&D) to translate business requirements into scalable data solutions
Required Qualifications
- 7+ years of hands-on experience with Databricks (Delta Lake, Spark, SQL, Python)
- Proven experience implementing medallion architecture (Bronze Silver Gold) at scale
- Strong expertise working with unstructured data parsing, transforming, and curating documents, PDFs, and text into structured data models
- Experience building data pipelines that feed AI/ML agents or LLM-based applications
- Working knowledge of Claude (Anthropic) and GPT (OpenAI) model integration, prompt engineering, or agent orchestration
- Proficiency in Python and SQL; experience with Databricks notebooks, workflows, and jobs
- Solid understanding of data modeling best practices for analytics and AI consumption
Compensation: The hourly rate for this position is between $80.00-$85.00 per hour.
Factors which may affect starting pay within this range may include [geography/market, skills, education, experience and other qualifications of the successful candidate].
Benefits: Sunrise offers ACA compliant medical coverage/dental insurance/vision insurance to all employees. We also offer Sick time benefits as required per State regulations.