Data Platform Engineer H/F

Détail du poste

What they need

Context

1.1 Business Context and purpose of the service
To increase its competitiveness and pursue its development in the best conditions, the significantly invests in its Digitalisation and Data Management, which undergoes many different transformation projects simultaneously. In this context, the team is structuring its data platform on Databricks, based on data governance and platform standards defined internally by the teams (data catalog structure, domain organization, access and usage rules).
The team is in charge of managing transversally and globally the Wind, Solar and Hydro tools in close relationship with a leading energy digital company and the IS of Countries and is looking for a Data Platform Engineer.
The service shall be carried out within the Delivery team, to support on data governance and on the management of its Databricks platform, and to lead the migration of its data from its legacy enterprise data lake to Databricks.

The objectives of the service are as follows:

- Administer and secure the Databricks platform of the team (technical and operational management)
- Deploy and enforce the data governance as applied to Databricks, and the related best practices (catalog structure, domain organization, access and usage rules)
- Drive the migration of data from the legacy enterprise data lake to Databricks
- Onboard and support data engineers and data citizens in using the platform
- Take cybersecurity, compliance and cost control into account "by design"
- Contribute to the 's IT community (exchanges on best practices, REX, etc.)

1.2 Services expected from the Service Provider
The activities presented above correspond to the expectations.
The Service Provider is expected to put forward a profile with the skills requested in line with the requested activities. After analysing the activities, the Service Provider may propose complementary skills.
All of the deliverables shall be drawn up according to the presentation models and standards in force .

1.3 Activities & Deliverables

The main activities are:

- Databricks platform administration: Administer, monitor and maintain the Databricks workspaces (technical and operational management within Databricks: workspaces, compute, access, cost, security, incident handling)
- Data governance on Databricks: Implement and evolve the data governance framework on the Databricks platform (catalog structure, domain organization, access rules, naming and quality standards) and ensure compliance with Group policies
- Legacy data lake migration: Lead the migration of data from the legacy enterprise data lake to Databricks: assess the existing perimeter, define the target design and roadmap, plan and coordinate the migration waves with the data teams, and secure the cut-over
- User onboarding and support: Welcome and support new users, from data engineers to simple data citizens, in adopting Databricks and in applying best practices; provide documentation, coaching and training
- DataOps on Databricks: Design and implement the pipelines that promote data assets and code across Databricks environments (dev, test, prod) and catalogs, using Git-based CI/CD and Databricks-native tooling (e.g. Databricks Asset Bundles), and promote DataOps practices among the teams
- Technology watch: Test and evaluate new Databricks features (including AI capabilities) and complementary tools, and recommend their adoption
- Reporting and communication: Report on platform and migration progress, and share best practices and feedback with the IT community

The main challenges are:

- Governance adoption: Getting entities and users to adopt the catalog, domain and access standards defined, while keeping the platform simple to use for citizens
- Migration complexity: Migrating legacy data lake data and consumers to Databricks without disrupting ongoing business usage, with data quality and lineage preserved
- Balancing run and build: Ensuring platform stability, security and cost control while delivering the migration and supporting new use cases

The main interfaces are:

- Digital Delivery teams (Data, Cloud and Business App)
- Data engineers and Data Citizens from the entities
- IS leads and business representatives
- Group Databricks CoE and Data architects
- Cybersecurity and Data Governance teams

The expected Deliverables are:

- Governance & operating model documentation: Catalog and domain structure, access model, naming conventions and usage standards aligned with the internal data and AI standards.
- Legacy data lake to Databricks migration plan and progress reports: Scope, prioritization, schedule, risks, and regular status reporting to the steering committees.
- Onboarding & best-practice material: Guides, templates, blueprints and training sessions for data engineers and data citizens.
- Platform run reports: Monitoring, cost, security and compliance indicators of the Databricks platform.

1.4 Expected skills

Skills Min Level (*) Importance Compulsory General skills & soft skills Communication and stakeholder relationship Confirmed 8 x Agile mindset (Scrum, Kanban) Confirmed 7 Analytical skills and autonomy Confirmed 8 x Teamwork, rigor and organization Confirmed 7 French Advanced 10 x English Advanced 8 x Functional skills Data governance applied to Databricks (Unity Catalog, domains, access management) Advanced 9 x Renewables business 3 Technical skills Databricks (administration, workspaces, Unity Catalog) Advanced 10 x DataOps on Databricks (environment & catalog promotion, Asset Bundles) Confirmed 9 x Git, CI/CD Confirmed 8 x Python, Spark Confirmed 8 x Cloud basics: AWS / Azure Junior 4 Data engineering (pipelines, data modeling) Confirmed 7 Terraform (Databricks provider: resources & permissions deployment) Junior 5 AI / GenAI knowledge (Databricks AI, LLM use cases) Junior 4 Infra / Network, Fabric, SMUS Junior 3

Compétences

- Databricks: 9 (Senior, 8 years
- Aptitudes à la communication (Écrite et Orale), assertivité: 7 (Senior, 8 years
- Data Management Systems: 7 (Advanced, 5 years

Le profil recherché

Profile wantedAdvanced skills in Data governance applied to Databricks (Unity Catalog, domains, access management)Advanced skills in Databricks (administration, workspaces, Unity Catalog)Confirmed skills in DataOps on Databricks (environment & catalog promotion, Asset Bundles)Confirmed skills in Git and CI/CDConfirmed skills in Python and SparkConfirmed skills in Data engineering (pipelines, data modeling)Junior skills in Cloud basics (AWS / Azure)Junior skills in Terraform (Databricks provider: resources & permissions deployment)Junior knowledge of AI / GenAI (Databricks AI, LLM use cases)Confirmedcommunication and stakeholder relationship skillsConfirmedanalytical skills and autonomyConfirmedteamwork, rigor and organizationAdvanced proficiency in French and English

Mais aussi...

Daily rate:Salary according to profile