Freelancer profile translated to English.

Description

Data engineer with 8 years of experience, I am looking for a long-term mission. I have experience in several sectors: media (FranceTV, TF1, RTL), transport (Ouigo, blablabus), retail (chantelle, diptyque), restaurant (burgerKing, Buffalo Grill), real estate (sergic, foncia), sport (FFF), energy (EDF)

Industry field of expertise

Languages

French
Fluent
English
Fluent
Spanish
Basic
Arabic
Native or bilingual

Workplace preferences

Can work on-site

Paris (up to 50km), Bezons (up to 50km)

EDF
GCP Data Engineer
ENERGY AND UTILITIES
September 2023 - Today (2 years and 11 months)
Chatou, France
OBJECTIVES
Migrate the current DataLake on Hadoop Hortonworks to Google Cloud Platform

MISSIONS
● Identify data sources and use cases in the current Data Lake
● Define data recovery methods by data source
● Participate in the formalization of the DAT (Technical Architecture Document)
● Identify GCP services for the different processing phases
● Industrialize GCP resources via Terraform (Datastream, GCS, Composer, BigQuery, ..)
● Develop Airflow ingestion DAGs (HTTPS, SFTP)
● Add the reverse-proxy part for the connection between databases and GCP
● Transform existing code (Hive, Hbase, Sqoop, Nifi) to prepare PySpark script
● Perform reverse engineering using Codey from Vertex AI service
● Develop DAGs to orchestrate the launch of PySpark scripts on Dataproc

TECHNICAL ENVIRONMENT
Python, Spark, pySpark
Google Storage, BigQuery, Dataproc serverless, Datastream, IAM
Terraform, Airflow
Confluence, Jira, SAFE, GitLab, GitLab CI/CD
Cloud GCP Spark PySpark Python Big Query Storage Dataproc Terraform Airflow Gitlab Gitlab CI/CD Atlassian JIRA SAFe
France Télévisions
GCP Data Engineer
FILM AND AV
February 2021 - August 2023 (2 years and 7 months)
Issy-les-Moulineaux, France
● Participate in the different phases of a project from analysis to deployment
● Build and monitor data processing pipelines in their entirety on GCP
● Participate in data governance by implementing MDM processes: deduplication, historization, calculation of indicators and aggregates...
● Collect data streams and implement quality controls and anomaly detection
● Clean and transform data from different non-homogeneous sources
● Optimize data processing and refactor code
● Develop new connectors (custom FacebookAds, GoogleAds operator)
● Implement RGPD treatments: anonymization and right to be forgotten
● Industrialize the creation of tables and views on BigQuery with Dbt
● Upgrade our internal tools (migration to Airflow 2 and Spark 3)
● Implement a new LakeHouse architecture
● Implement real-time processing jobs for certain data sources (PubSub, Dataflow)
● Maintain and ensure code quality and promote best practices within the team (testing, CI/CD, code review, ..)
● Implement a profiling solution (Python, Cloud Function, Airflow, Dataproc)
● Contribute to and ensure the technical documentation of projects on Confluence
● Maintain permanent technological watch in the field
● Prepare a Quickstart for new joiners
● Participate in the onboarding phase
Python 3 Spark Google Cloud Platform (GCP) Big Query Google Storage Dataproc PubSub Cloud Run Cloud transfer Cloud Functions Apache Airflow DBT Terraform
Leadeal Marketing
Data engineer
DIGITAL AND IT
October 2018 - January 2021 (2 years and 4 months)
Paris, France
● Development of an API for an internal application in Spark/Scala
● Optimization and addition of new features to the KAMINO tool (internal company ETL)
● Collection of data from third-party applications or information systems (relational databases,
API, MongoDB, Azure, AWS, FTP, ...)
● Implementation and automation of data processing pipelines (data streams, Data
Lake, data ingestion and structuring, transformation and indicator calculation)
● Cleaning and deduplication of collected data
● Translation of functional requirements from the business team into technical solutions
adapted to different projects
● Deployment and planning of workflows in production environments using
Airflow
● Design, creation and maintenance of relational databases (reporting Datamart,
customer file Datamart, PRM/CRM databases)
● Implementation of the anonymization process within the framework of the General Data Protection Regulation (GDPR)
● Operation and monitoring of daily processing, investigation and correction of anomalies
● Implementation of unit tests
● Automation of deployment using GitLab CI/CD
● Knowledge sharing within the team and establishment of best practices
● Documentation of workflows on Confluence
● Technological watch
Scala Spark Java SQL Server MSSQL AWS S3 EMR AWS Azure Storage Azure HDInsight Microsoft Azure Apache Airflow

Be the first to recommend Mohamed Raed

Help this freelancer shine by sharing your experience working together.

Agatha Frydrych

Backend Java Software Engineer

4.7

(3)

Baptiste Duhen

Fullstack developer

4.6

(4)

Amed Hamou

Senior Lead Developer

(2)

Audrey Champion

Web developer

4.3

(3)

Signup to reveal

Master
UPMC, Paris, France
2018
Engineering Degree
Ecole Nationale d'Ingénieurs de Sousse, Tunisia
2018

GCP Professional Data Engineer
Google
2023
https://google.accredible.com/1656bd42-2a40-4ed8-bbbe-869584a443ba
Datastream Big Query Identity and Access Management (IAM) Google Composer Google Cloud Storage Google cloud Dataproc Dataflow

Data Engineer

Cloud Engineer & Architect

Mohamed Raed A.

GCP Data Engineer

About Mohamed Raed

Experience

Recommendations

These freelancer profiles also match your criteria

Education

Certifications

Skill set

Categories