Freelancer profile translated to English.

Description

As an Azure Data Engineer, I integrate various data sources to meet advanced analytical needs. These sources are ingested and stored in Azure Data Lake storage accounts. These files are then used by multiple use cases where I generate business views using PySpark in Databricks, with the entire process orchestrated by Azure Data Factory.

In data exploration mode, within Databricks, I create notebooks to generate Delta tables in the Hive Store or files in DBFS. These notebooks can be scheduled to automate processing.

Working with Data Scientists, I prepare and clean datasets for Machine Learning needs.

Passionate about acquiring new skills, I remain open to upskilling on new tools or other platforms.

Industry field of expertise

Languages

French
Native or bilingual
English
Fluent

Workplace preferences

Can work on-site

Paris (up to 50km)

AXA GO
Senior Data Engineer
November 2024 - Today (1 year and 9 months)
Paris, France
Creation of data ingestion pipelines for data landed in the Data Lake.
Data transformation using Pyspark/Scala to produce business-oriented views.
Orchestration of jobs using Azure Data Factory
Management of the data lifecycle in the lake.
Code optimization for cost control in Azure
Azure cost monitoring
Data refresh in Azure Analysis Services cubes or Datasets.
Power BI report creation
Assistance in writing technical specifications
Execution of unit and integration tests
PySpark Azure Databricks Azure Data Factory Azure DevOps Scala
AXA en France
Data Engineer
February 2020 - September 2024 (4 years and 8 months)
Île-de-France, France
Mission:
Creation of data ingestion pipelines for data landed in Azure Data Lake from various data sources and generation of Business Views in the Azure Cloud environment.
Implementation of an Azure Analysis Services cube

Work Organization:
I work as a Data Engineer and Data Analyst
Gathering and analyzing business needs and proposing technical solutions
Defining interface contracts with internal and external partners

Roles:
Ingestion of raw data and integration into the Data Lake in PARQUET format.
Creation of the SILVER layer (historized) from prepared data in DELTA format.
Generation of the GOLD layer by creating Business Views to meet specific business needs.
Creation of SQL tables in Azure Databricks work environments to meet Power BI Reporting needs.
Export of data from USE CASES to mount points accessible by partner applications.
Creation and maintenance of Azure Data Factory pipelines.
Analysis and resolution of Big Data environment incidents.
Validation of Business Views and debugging of Pyspark code
Execution and formalization of unit tests and integration tests.
Code management and continuous integration on Azure DevOps.
Writing detailed technical specifications.
Data security in the Cube.
Creation and management of Azure Pipelines.
Translating business needs into PySpark to create Business Views.
Developing Business Views in Pyspark
Technical documentation.
Data exploration using Oracle SQL Developer (main source)
Developing Azure Analysis Services cubes
Developing Power BI reports and Dashboards
Deployment and Go-Live of deliverables as Tech Lead.
Technical coordination for defining technical architecture
Agile tracking of subject progress
PySpark Azure Databricks Azure Data Lake Azure Data Factory Azure DevOps
Société Générale
MS BI Consultant
August 2019 - January 2020 (5 months)
Île-de-France, France
Mission:
Implementation of a monitoring project for instances and licenses of servers and Middleware applications.
Loading and transformation of source data (STG, ODS, and DWH) and creation of Power BI reports.

Work Organization:
I work as a Technical-Functional Consultant for writing functional and technical specifications.
Gathering and analyzing business needs.

Roles:
Writing detailed technical specifications.
Defining technical architecture.
Technical documentation.
Development across different layers:
- ETL (SSIS): For loading data from various operational systems databases into Staging area, ODS, and Data Warehouse.
- Creation of Power BI reports and dashboards.
Design and Creation of Staging, ODS, and Data warehouse databases.
Creation of Stored Procedures for generating loading logs.
Execution and formalization of unit and integration tests.
Configuration management using GitHub.
Deployment and Go-Live of deliverables.
Requesting VM creation and associated access rights.
Technical coordination.