Job Description: Senior Data EngineerRole Summary
We are looking for a Senior Data Engineer with strong hands-on experience in Azure Databricks, PySpark, and modern data engineering practices. This role focuses on building scalable data pipelines, optimizing data workloads, and supporting Lakehouse-based solutions in a banking and financial services environment.
Key Responsibilities
Develop and maintain batch and streaming data pipelines using Azure Databricks, PySpark, Delta Lake.
Implement data ingestion using Auto Loader, ADF/Synapse, Event Hubs/Kafka.
Build high quality ETL/ELT workflows for financial datasets (payments, transactions, customer, fraud, regulatory).
Apply Delta Lake best practices: schema evolution, versioning, CDC, and data quality checks.
Optimize Spark jobs (partitioning, caching, Z ordering, AQE, compaction).
Ensure secure and compliant data handling using Unity Catalog/Purview, RBAC, and encryption.
Work closely with architects, data scientists, and business teams to deliver reliable data solutions.
Contribute to DevOps pipelines using Azure DevOps, CI/CD, and Infrastructure as Code.
Participate in code reviews, troubleshooting, and performance tuning.

Mandatory Skills
Strong hands-on experience with Azure Databricks, PySpark, Delta Lake, Structured Streaming.
Knowledge of Azure data tools: ADF, Synapse, ADLS Gen2, Event Hubs/Kafka.
Solid SQL/NoSQL skills and understanding of data modeling.
Experience with Spark performance tuning and large-scale data processing.
Familiarity with data governance tools (Unity Catalog / Purview).
Experience with Git, CI/CD, and IaC (Terraform/ARM/Bicep).
Good communication and documentation skills.

Preferred Skills
Experience in banking/financial services (fraud, risk, payments, regulatory reporting).
Exposure to multi-cloud Databricks (AWS/GCP).
Understanding of event-driven architectures and microservices.
Experience supporting ML workflows (Feature Store, MLflow).