Overview:

Machine Data (MDC): MDC team delivers Data Products which fuel mission critical use cases, e.g. Fleet Diagnostics, Process Efficiency, Availability Reporting and others. They process data coming from machines installed worldwide (around 3500 machines sending data daily). Currently they are focused on migrating their workloads from legacy on-prem environment to ADAP on Azure.

A candidate will be responsible for maintaining existing systems and implementing future changes as client consumption needs and ingestion evolve, becoming larger and more extensive.

What will you do?

• Cloud spendings;
• Pipelines performance;
• Pipelines orchestration;
• Data monitoring;
• Understanding of impact from upstream data quality issues on downstream datasets on records level, w/o explicit DQ checks;
• Maintain and enhance existing data systems;
• Develop enhancements for the existing Data Platform;
• Implement future changes to accommodate growing consumption needs and ingestion;
• Work with complex datasets, including the addition of new datasets from scratch;
• Collaborate with senior team members on complex projects, particularly when adding new large datasets;
• Grow with the technological landscape, adapting to new tools and methodologies as they emerge.

What are we looking for?

The customer is looking for a Data Engineer to extend the team capacity:

• Development Support of existing pipelines: Resolving new/complex data issues and root cause analysis in the Generated Business Record (Gold layer);
• Implementation of new demand coming from Data Consumers or from Platform team (e.g. in case of changes in the Azure DevOps stack);
• Proficiency in PySpark;
• Knowledge of the Azure cloud stack;
• Experience in working with large and complex datasets;
• Ability to work in a team and collaborate with various stakeholders;
• Experience in Azure Analytics Engineering;
• Familiarity with DevOps practices.