Ready to build the future with AI?
At Genpact, we don’t just keep up with technology—we set the pace. AI and digital innovation are redefining industries, and we’re leading the charge. Genpact’s AI Gigafactory, our industry-first accelerator, is an example of how we’re scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to agentic AI, our breakthrough solutions tackle companies’ most complex challenges.
If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of what’s possible, this is your moment.
Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge, operational excellence, and cutting-edge solutions – we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation, our teams implement data, technology, and AI to create tomorrow, today. Get to know us at genpact.com and on LinkedIn, X, YouTube, and Facebook.
Inviting applications for the role of Lead Consultant, ADF and Airflow Engineer
In this role Develop and implement complex data transformation logic and workflows using Python, PySpark, and SQL within the Azure ecosystem (e.g., Azure Databricks, Azure Synapse Analytics).
Responsibilities
Data Pipeline Architecture Development:
Design, build, and maintain scalable and reliable ETL/ELT pipelines using Azure Data Factory (ADF) to ingest, process, and transform large volumes of structured and unstructured data.
Develop and implement complex data transformation logic and workflows using Python, PySpark, and SQL within the Azure ecosystem (e.g., Azure Databricks, Azure Synapse Analytics).
Utilize Apache Airflow for the robust orchestration, scheduling, and monitoring of these complex data pipelines and workflows.
Performance Optimization Tuning:
Optimize data processing jobs and queries for performance, scalability, and cost-effectiveness in Azure environments.
Monitor pipeline performance, troubleshoot failures, and implement improvements to ensure high efficiency and reliability of data systems.
Perform advanced SQL tuning and work with big data technologies to manage and process large datasets efficiently.
Data Governance Quality Assurance:
Implement and enforce data quality checks, governance standards, and security best practices (e.g., encryption, access control) across all data pipelines and storage solutions.
Manage and optimize data storage solutions, including Azure Data Lake Storage (ADLS), Azure SQL Database, and Azure Synapse.
Collaboration Participation:
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into effective technical solutions.
Actively participate in solution design and architecture discussions, taking end-to-end ownership of data engineering initiatives.
DevOps Automation:
Implement CI/CD pipelines for deploying ADF pipelines and Airflow DAGs (Directed Acyclic Graphs) using tools like Azure DevOps or Jenkins.
Automate data workflows, error handling, and alerting mechanisms for production systems.
Qualifications we seek in you
Minimum Qualifications
hands-on/project experience in data engineering, specifically in data ingestion, integration, transformation, and pipeline orchestration. Used ADF and Airflow at a greater scale in multiple projects.
A bachelor’s degree or above in computer science, Information Technology, Engineering, or equivalent
In-depth knowledge of Azure Data Factory (ADF) capabilities, including designing, developing, and deploying complex ETL/ELT pipelines, linked services, datasets, and triggers.
Strong Proficiency in Apache Airflow for designing, developing, and maintaining scalable data pipelines and Directed Acyclic Graphs (DAGs).
Strong proficiency in Python, Pyspark for data processing, automation, and general programming tasks, along with expert-level SQL knowledge (including query optimization for large datasets).
Significant experience with the Azure cloud ecosystem, including services such as Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure Databricks, Azure SQL Database, and Cosmos DB.
Solid understanding of data warehousing, data modelling, database management, and ETL/ELT methodologies.
Strong working knowledge of code management (Git/GitHub) and continuous integration/continuous deployment (CI/CD) pipelines using tools like Azure DevOps
Ability to work independently or cross-functionally and communicate effectively with technical and non-technical stakeholders (e.g., analysts, business teams) is highly valued.
Owning and designing end-to-end scalable data architectures and complex pipelines.
Monitoring, troubleshooting, and optimizing data pipelines and platform performance.
Collaborating with cross-functional teams (analysts, data scientists, business teams) to gather requirements and deliver solutions.
Enforcing data governance standards and best practices.
Preferred Qualifications/ Skills
Experience with other languages like Scala or Java is also beneficial for big data environments.
Hands-on experience with Databricks for data processing and running data pipelines is a recurring requirement.
Experience with ADLS and Azure Synapse Analytics is crucial for comprehensive data solutions.
Familiarity with Azure Key Vault, Azure Logic Apps, Event Grid, and Azure Purview
Knowledge of data governance standards, security protocols, data quality implementation, lineage, and access control is a preferred qualification
Good understanding of the insurance domain and its functionality.
Good to have MS Azure DP-203 certified
Why join Genpact?
Lead AI-first transformation – Build and scale AI solutions that redefine industries
Make an impact – Drive change for global enterprises and solve business challenges that matter
Accelerate your career—Gain hands-on experience, world-class training, mentorship, and AI certifications to advance your skills
Grow with the best – Learn from top engineers, data scientists, and AI experts in a dynamic, fast-moving workplace
Committed to ethical AI – Work in an environment where governance, transparency, and security are at the core of everything we build
Thrive in a values-driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress
Come join the 140,000 coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up.
Let’s build tomorrow together.
Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation.
Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.