Job Title: Senior Data Engineer

Experience: 6+ Years

Key Responsibilities

Design, develop, and maintain scalable and reliable data pipelines using Azure services and Databricks.

Build efficient data ingestion and transformation workflows using Databricks, PySpark, and Delta Lake.

Develop, optimize, and maintain ETL/ELT pipelines using Python and SQL.

Implement best practices for data modeling, data quality, governance, and performance optimization.

Work with relational and lakehouse databases, including PostgreSQL and Delta Lake.

Monitor, support, and troubleshoot production data pipelines.

Collaborate with cross-functional teams including product, analytics, and engineering stakeholders.

Required Skills & Experience

5+ years of hands-on experience in Data Engineering.

Strong expertise in the Azure ecosystem, including:

Azure Databricks

Azure Data Lake Storage (ADLS)

Azure Functions

Azure Data Factory (preferred, not mandatory)

Strong experience with Apache Spark (PySpark).

Advanced SQL skills, including query tuning and optimization.

Strong understanding of ETL/ELT methodologies and data warehousing concepts.

Hands-on experience with Delta Lake (ACID transactions, schema evolution, time travel).

Experience working with PostgreSQL or similar relational databases.

Excellent problem-solving skills and attention to detail.

Strong communication skills with the ability to engage both technical and non-technical stakeholders.

Self-driven, proactive, and comfortable working in a fast-paced environment.

Preferred / Nice-to-Have Skills

Experience with CI/CD pipelines (Azure DevOps, Git).

Knowledge of Infrastructure as Code (IaC) tools such as Terraform.

Exposure to streaming platforms like Kafka or Azure Event Hub.

Familiarity with data governance tools, such as Azure Purview.