See all the jobs at Caixa Mágica Software here:
| Data | Full-time | Partially remote
Overview:
We are looking for a Senior Databricks Engineer with deep, hands-on expertise in Azure Databricks. While our team has experience with Azure Data Factory (ADF) and pipeline development, we are specifically seeking a senior-level engineer who can take ownership of all Databricks-related architecture, development, and optimization.
This role is essential to strengthen our Azure data platform and accelerate the delivery of high-quality, scalable data products into our Global Data Platform (GDP).
What will you do?
- Take the technical lead in all Databricks-related development and architecture;
- Design and implement scalable, high-performance data pipelines using Databricks notebooks (PySpark);
- Develop reusable frameworks, templates, and best practices for Databricks development;
- Collaborate with data engineers, architects, and analysts to translate business needs into robust Databricks solutions;
- Integrate Databricks with other Azure services such as ADF, Data Lake, and Azure SQL;
- Guide and mentor team members in Databricks usage, performance tuning, and modular development.
What are we looking for?
- Extensive hands-on experience with Azure Databricks in production environments;
- Strong proficiency in PySpark, SQL, and Delta Lake;
- Deep understanding of Lakehouse architecture, data modeling, and data quality frameworks;
- Familiarity with Collibra for metadata management and data governance integration;
- Experience with CI/CD pipelines using Azure DevOps, Git, and Infrastructure as Code;
- Familiarity with Azure Data Factory, Azure Data Lake, and Azure SQL;
- Ability to coach and support other engineers in Databricks best practices;
- Fluent in English; Dutch is a plus;
- Microsoft certifications such as DP-203 or Databricks Certified Data Engineer;
- Experience with metadata-driven development, data governance, and data mesh principles;
- Background in insurance, finance, or public sector environments is a strong advantage.
What can you expect from us?
- A permanent job contract for a long term project;
- Tech equipment + SIM Card + personal smartphone;
- Health and Life Insurance;
- Social events and team buildings;
- The commitment of letting you grow with us, and be rewarded accordingly;
- A dynamic and young team that will be always there to support you;
- Training in the latest technologies;
- Coffee, fruits, snacks and a warm welcoming when you pass by the office.