Senior Databricks Developer

We are looking for a Senior Databricks Developer responsible for designing, developing, optimizing, and operating data solutions on the Databricks platform. The role contributes to the implementation and evolution of the organizational Data Lakehouse, building data ingestion, transformation, processing, and delivery processes for reporting, advanced analytics, and AI/ML initiatives. You will have end-to-end ownership of data flows and act as a technical reference within the team.

Core Responsibilities

• Data Development and Integration: Develop and maintain data ingestion, transformation, and publishing processes using Databricks and PySpark; build and optimize ETL/ELT pipelines for large data volumes; implement Delta Lake structures and data models for operational and analytical consumption; develop incremental loading, historization, and CDC (Change Data Capture) mechanisms; integrate data from multiple sources (Oracle, SQL Server, APIs, files, banking applications, and more).
• Architecture and Data Modeling: Participate in defining the Data Lakehouse architecture; propose scalable, high-performance data models; contribute to development standards and platform architecture; ensure data traceability and consistency across the entire processing flow.
• Data Quality and Operations: Implement automated Data Quality controls; develop data reconciliation and validation processes; configure monitoring, alerting, and logging; analyze and remediate production incidents; manage reloads, reruns, corrections, and data recovery.
• Optimization and Performance: Optimize PySpark and SQL processes for cost and performance; identify and eliminate processing bottlenecks; optimize Databricks resource consumption; contribute to automating development and deployment processes.

Qualifications and Experience

• Minimum 5 years of experience in Data Warehouse, Big Data, or enterprise data platform development.
• Minimum 2-3 years of hands-on Databricks experience.
• Proven experience in complex Data Warehouse, Data Mart, or Data Lake projects.
• Experience building end-to-end data pipelines.
• Solid knowledge of Data Lakehouse architecture.
• Experience implementing historization, CDC, reconciliation, and Data Quality processes.
• Ability to take full ownership of technical deliveries and provide technical leadership to the team.

Tech Stack

• Mandatory: Azure Databricks / PySpark, advanced SQL, Delta Lake, Git & Jira, Data Modelling, ETL/ELT, Data Quality Frameworks, Pipeline Orchestration.
• Nice to have: Azure Data Factory, Kafka, Event Streaming, advanced Python, CI/CD, Unity Catalog, Data Governance, Machine Learning on Databricks, Qlik Sense.

Banking Experience (Major Advantage)

• Knowledge of Retail Banking products and processes.
• Experience with financial and management reporting.
• Knowledge of risk, compliance, and regulatory reporting processes.
• Understanding of banking data flows and enterprise analytics ecosystems.