Position Summary

This role involves leading end-to-end delivery of modern data engineering solutions, combining hands-on technical expertise with delivery and team leadership. The role is responsible for designing, building and migrating enterprise-scale data pipelines from legacy platforms (Informatica BDM, TIDAL) to modern cloud-native architectures (Databricks, Apache Airflow), while ensuring data quality, governance and operational stability. This role serves as a critical link between architecture, engineering and delivery teams, ensuring high-quality, timely execution of data initiatives across global programs and orchestrating high-quality data engineering practices with governance adherence.

Job Responsibilities Lead design, development and implementation of scalable data pipelines on Databricks. Build and migrate data pipelines from Informatica BDM to Databricks. Migrate job scheduling from TIDAL to Apache Airflow. Develop data processing logic using PySpark, SQL and Python. Ensure pipelines are reliable, fast and easy to support. Build and manage Airflow DAGs for job scheduling and monitoring. Follow best practices while working with Databricks and Delta Lake. Add data quality checks and basic reconciliation controls. Maintain clear documentation such as designs, mappings and runbooks. Guide and review the work of data engineers. Help break down work into tasks and estimates. Track progress using JIRA and Confluence. Support sprint planning, reviews and delivery tracking. Serve as a technical SME for commercial pharma datasets. Mentor junior engineers and enforce coding and delivery best practices. Coordinate with BAs and PMs for alignment of business priorities with tech delivery. Education BE/B.TechMaster of Computer ApplicationWork Experience 8–11 years of experience in data engineering, with 2–3 years in a Technical Lead / Manager role. Strong hands-on experience with Databricks (PySpark, Spark SQL, Delta Lake), Informatica BDM and Informatica IICS / IDMC, Apache Airflow, SQL, Python Proven experience in Informatica BDM → Databricks migration initiatives. Proven experience migrating job scheduling/orchestration from TIDAL to Airflow. Experience with CI/CD pipelines, Git-based version control and environment promotion strategies. Strong understanding of data pipeline performance tuning, error handling and operational support. Experience managing Agile delivery, sprint backlogs and delivery tracking using JIRA. Experience working on large scale data modernization or cloud migration programs. Familiarity with pharma data standards, KPIs and reporting constructs. Prior experience in customer-facing global delivery engagements. Exposure to multi-vendor or onshore-offshore delivery models. Knowledge of data reconciliation frameworks and audit readiness in regulated environments. Behavioural Competencies Teamwork & LeadershipMotivation to Learn and GrowOwnershipCultural FitTalent ManagementTechnical Competencies Problem SolvingLifescience KnowledgeCommunicationProject ManagementCapability Building / Thought LeadershipInformaticaDatabricksSkills