Freelancer profile translated to English.

Description

Lead Big Data / Data Engineer for 7 years. Currently looking for a new opportunity on Scala/Spark/Kafka technologies... I also have skills in cloud (AWS, GCP)

For the past 7 years, I have mainly worked on the design and maintenance of applications for real-time data processing and reactive architecture on one hand, and on batch processing with ETLs on the other.

Languages

Arabic
Native or bilingual
English
Fluent
French
Native or bilingual

Workplace preferences

Can work on-site

Paris (up to 50km)

Société générale
Lead Data Engineer / Business Analyst
BANKING AND INSURANCE
January 2024 - Today (2 years and 5 months)
Nanterre, France
My responsibilities:
• - Guarantee of good practices, code review, documentation
• - Technical and functional referent
• - Automation of Spark, SQL deployments (CI/CD)
• - Continuous improvement related to BUILD and RUN (N1/N2/N3 support)
• - Share Scala/Spark best practices with other teams
• - Migration from Spark 2 to Spark 3
• - Automation of Spark jobs with Oozie
• - Exposure of data via Hive views
• - Optimization of SQL queries to improve data processing performance
• - Implementation of a data quality solution to ensure the reliability of data used in analyses,
SQL Scala Spark Oozie Hive
PMU
Lead Data Engineer
ENTERTAINMENT AND LEISURE
January 2022 - January 2024 (2 years)
Paris, France
My responsibilities:
• - Ingestion of raw data with AWS Glue.
• - Technical design
• - Monitoring of metrics and Spark jobs activity via Kibana.
• - Format conversion according to business needs (renaming, enrichment, harmonization).
• - Writing Athena queries for data exploitation and control
• - Data anonymization,
AWS Glue AWS S3 AWS EC2 Spark Scala
La Poste colissimo
Data Engineer
January 2019 - January 2022 (3 years)
My responsibilities:
• - Guarantee of good practices, code review, documentation
• - Technical and functional referent
• - Optimizations of Spark Streaming / Akka Stream processing
• - Automation of deployments (CI/CD)
• - Participation in the integration of new products into the IS in collaboration with the technical architect.
• - Contribution to the development and validation of the project schedule.
• - Responsibility for compliance with quality standards on delivered products.
• - Participation in the definition of an architecture for monitoring and planning transport.
• - Design and development of a VAT calculation engine with Spark, Kafka and PostgreSQL.
• - Development of administration APIs for Cassandra and PostgreSQL databases with Play Framework.
• - Improvement of data quality by avoiding the loss and merging of packages through the redesign of an Akka API.
• - Development of a tool for extracting and visualizing API logs in Kibana using Elasticsearch.
• - Participation in the refactoring of Scala applications by applying the best practices of the language.
• - Creation of Docker images to set up a virtual architecture dedicated to testing.

Technical context: Scrum, Spark Streaming, Kafka, Cassandra, Postgres, Play API, Linux, Git, Agile methods, SQL, Elasticsearch, Kibana, Kubernetes, Docker, Jenkins VTOM, Technical leadership.
Docker Scala Elasticsearch Apache Kafka PostgreSQL