Job Description Insight Global's client is seeking a highly skilled AI Data Engineer to join their Data Office AI Team. In this role, candidates will lead data strategy and execution for AI initiatives, focusing on the sourcing, enrichment, and optimization of data for Microsoft Azure-based AI systems. You will be the technical expert ensuring that data quality, structure, and metadata are optimized for high-performance search and retrieval-based AI (RAG). While AI Engineers focus on model logic and prompting, this role will own the data foundation, building the automated pipelines and semantic structures that make enterprise AI effective. This role operates in a modern, AI-augmented development environment. Engineers are expected to leverage agentic coding tools (e.g., Cursor, Claude, Copilot) to accelerate development, automate pipeline generation, and build self-improving data systems. Manual, static pipeline development is the exception and not the norm. Key Responsibilities include: AI Data Architecture & Optimization -Semantic & Retrieval Design: Architect data models and semantic structures (RDF, property graphs) and lead optimization of Azure AI Search (Vector/Hybrid) to improve RAG performance. -Unstructured Data Architecture & Strategy: Design chunking, embedding, and metadata strategies for complex content (PDFs, transcripts) to ensure RAG-ready datasets that drive high-accuracy retrieval. -Evaluation & Observability: Implement RAG evaluation frameworks (Ragas, TruLens) to measure groundedness and retrieval precision, using these metrics to trigger automated architectural refinements. Agentic Engineering & DataOps -Self-Improving Pipelines: Build autonomous pipelines that use feedback loops to refine transformations and metadata over time. Leverage agentic coding to generate, test, and refactor code. -Modern Data Stack: Utilize Microsoft Graph APIs and the full Azure stack (Synapse, Data Lake, Functions) to build scalable, parallel-execution (PAR) workflows. -Resilience: Apply DataOps for agent-driven testing and proactive detection of data drift or retrieval degradation. Strategic Business Partnership & Governance -AI-Readiness Consulting: Engage stakeholders to clarify requirements and articulate when data is "not AI-ready" due to poor metadata or governance gaps. Set realistic timelines for enrichment. -Governance Coordination: Align all AI projects with CDO policies, regulatory standards (GDPR, SOC2), and security protocols. Proactively identify opportunities to integrate AI initiatives with existing enterprise programs. -Leadership & Mentorship: Serve as a technical bridge, translating complex data concepts for business groups while providing hands-on mentorship to junior AI data engineers. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/. Skills and Requirements -5+ years in data engineering -Proven track record of delivering AI or search-based solutions. -Advanced Python skills -Advanced Typescript -Deep experience with the Azure Data & AI stack (Azure AI Search, ADF, Azure OpenAI). -Experience with AI-Augmented Workflows: Expert-level use of AI coding assistants (Cursor, Claude, GitHub Copilot) to generate production-grade code and tests. -Strong understanding of Information Architecture, graph-based structures, metadata strategies, and taxonomy. -Ability to explain data readiness and technical impacts to non-technical stakeholders (HR, Marketing, Ops). -Masters Degree in Data Science, ML or Artificial Intelligence -Data Center background