Freelancer profile translated to English.

Description

Big Data Engineer with multi-sector experience (retail, reinsurance, telecommunications). Very familiar with Big Data (spark, Hadoop, kafka), cloud computing (Azure/AWS), especially Azure, and software development. I obtained a Spark Databricks 3.0 certification.

Industry field of expertise

Languages

English
Fluent
French
Fluent
Spanish
Basic

Workplace preferences

Can work on-site

Paris (up to 50km)

ENEDIS
Tech Lead DATABRICKS
ENERGY AND UTILITIES
June 2024 - Today (2 years and 2 months)
Lead functional and technical scoping of the project
Implement a medallion architecture
Implement data ingestion and transformation with Databricks and Pyspark (python)
Implement a mechanism to ensure data quality and control
Create interface contracts with source and destination data and use them for data controls
Orchestrate workflows with Databricks lakeflow (jobs, pipeline)
Implement the CI/CD chain with Databricks asset bundle and GitlabCI
Create star schemas and expose data in Fact and Dimension tables
Calculate KPIs and present them in dashboards for the business
Azure Databricks Python SQL Server DevOps Lakehouse
AXA GO
Consultant Azure Data Engineer
May 2022 - June 2024 (2 years and 1 month)
Scope and prepare functional and technical specifications.
Support coordination with third-party teams (data integration team and data management team).
Implement data ingestion and transformation. Implement specific calculations and filtering based on needs.
Design the technical solution covering the needs (data model, integration flows, data mapping).
Build an integration layer based on Azure Datalake: DAX, Databricks, Azure Cubes, Azure DevOps, Azure Analysis Services. PowerBI dashboards connected to Azure Cubes.
Implement specific calculations and filtering based on needs.
Technical environment: Spark, Databricks, Scala, Python, Azure Data Factory, AzureDataLake, Azure DevOps, Azure Analysis Services, Teradata, SQL, PowerBI, ServiceNow
Azure Data Factory Azure Databricks Azure Data Lake PowerBI Azure DevOps
FNAC Darty
Azure Data engineer
LOGISTICS AND SUPPLY CHAIN
January 2021 - April 2022 (1 year and 3 months)
Paris, France
Project: FNAC/DARTY brands' HR data normalization and centralization platform in a common Data Warehouse.
Achievements:
•Automate the daily feeding of the system with source files into the DataLake
•Develop data-driven workflows scheduled daily and monthly with Azure Data Factory
•Parse and clean source files with Spark Databricks
•Calculate indicators and history with Spark Databricks
•Apply transcoding (mapping with repositories)
•Store results for use in Azure DataWareHouse
Microsoft Azure Databricks Azure Data Factory Scala Data Lake