Skills/Experience:
- Design and development of ELT pipelines using Azure Data Factory and Databricks
- Optimisation and maintenance of data workflows, ensuring quality and integrity
- Performance tuning and monitoring
- Data Modelling / Power BI semantic modelling
- Notebook development using Python, SQL and/or PySpark
- Implementation of best practices for data engineering, including governance and security
- Extensive experience building data pipelines in a Databricks Lakehouse environment
- Well-versed with Spark and other big data technologies
- Excellent coding skills using SQL and Python
- Strong background developing solutions in an Azure Cloud environment, including strong ADF development skills
- Data Modelling skills
