Data Engineer – AI

Crossing Hurdles is seeking a Data Engineer specializing in AI to design, implement, and optimize AI/ML models for production-grade applications. The role involves developing multi-agent systems, integrating solutions on secure cloud environments, and collaborating with various teams to ensure high-quality data for AI training and inference.

Responsibilities:

• Design, implement, and optimize AI/ML models using large language models (LLMs), retrieval-augmented generation (RAG), and prompt engineering for production-grade applications
• Develop and orchestrate multi-agent systems utilizing frameworks such as LangGraph and LangChain
• Integrate and deploy solutions on secure cloud environments, including AWS GovCloud, Google GovCloud, Azure IL5+, Vertex AI, and AWS Bedrock
• Build robust data pipelines, manage ETL processes, and develop metadata catalogs and ontologies to ensure high-quality data for AI training and inference
• Create and maintain REST APIs and SDK integrations to facilitate seamless data and model interactions
• Collaborate with product, security, and engineering teams to ensure best-in-class delivery, adhering to secure coding and DevOps best practices

Requirements:

• Have strong relevant experience in Python for AI/ML development, including proficiency with REST APIs and SDK integration
• Have hands-on experience with LLMs, RAG systems, and prompt engineering in production environments
• Be familiar with multi-agent orchestration, tool use, and frameworks like LangGraph and LangChain
• Possess a deep understanding of cloud AI services, including AWS GovCloud, Google GovCloud, Azure IL5+, Vertex AI, and AWS Bedrock
• Have a background in building and maintaining data pipelines, ontologies, metadata catalogs, and ETL processes