Freelancer profile translated to English.
Back to original languageAbout Malik
As a data engineer, I am capable of intervening across the entire data processing chain, from the research phase to the industrialization process. During my various experiences, I have encountered diverse and varied problems. Therefore, I believe I now have enough perspective to best support clients on different technical issues effectively and relevantly.
I have always worked in mixed teams of about ten people, with other data engineers and data analysts or data scientists.
In terms of technologies I have had the opportunity to work with during my experiences as a data engineer:
- HDFS, Gitlab CI, Logstash, ElasticSearch.
- pySpark, pandas, Kafka, AWS (Kinesis, S3, RDS, ECS, Cloud Watch)
- Airflow, Scala, Google Cloud platform (BigQuery, Google Cloud Storage, Data Flow,).
- Sap BO, Kibana.
- Sql alchemy, MongoDB, Oracle
French
Native or bilingual
English
Native or bilingual
Arabic
Native or bilingual
German
Basic
Italian
Basic
Kabyle
Native or bilingual
Can work on-site
Paris (up to 50km)
Experience
- bpifranceData EngineerBANKING AND INSURANCESeptember 2022 - Today (3 years and 9 months)Maisons-Alfort, FranceAs a data engineer within the IT department, specifically within the Finance and Risk department, my tasks involve managing the technical aspects related to data collection, processing, storage, and analysis to meet the business needs of the finance department. My main tasks are summarized as follows:- Implementation of data ingestion and transformation flows using Pyspark, from the raw compartment to the compartment managed by our department (trusted).- Implementation of Glue jobs to aggregate data using Pyspark, from our internal department, to make them available in the MOU compartment.- Creation of Datadog monitoring dashboards to ensure the monitoring and performance of ingestion and data aggregation tasks, monitoring the performance of Glue jobs, and detecting errors.- Development of Lambda triggers responsible for initiating Glue jobs when new data arrives in an S3 bucket. These triggers also allow for reprocessing historical data that arrived late.- Improvement of code coverage by performing regression tests using pytest.- Migration of projects to a multi-tenant architecture.- Making MOU data available via an API for business users, using the API Gateway service for authentication, authorization, and security of the exposed data.- Obtaining GoProd and GoDataset accreditations for my team.
- ENGIE ITCloud Data EngineerENERGY AND UTILITIESSeptember 2021 - August 2022 (1 year)Bagneux, FranceSeptember 2021 – Present Data Engineer, Engie ITWithin the middleware center of expertise, I am responsible for developing tools for centralizing data from various providers (Electricity and Gas) for the group's business teams.- Implementation of a data batch processing pipeline for ingesting raw data (flat files) in staging and series steps using Spark and Scala to the Data Hub on an S3 bucket.- Packaging Scala-Spark jobs into JAR files using Maven, running on a Databricks cluster.- Using Spark with Python to distribute data processing on large datasets, significantly improving ingestion speed.- Implementation of a scheduler (Airflow) to automate Spark jobs with a daily frequency.- Migration of Kafka connectors to the Kinesis reception bus.Querying and analyzing Delta data via AWS Athena.- Implementation of a customer data history deletion automation application for gas and electricity meter readings (at the customer's request and within the framework of the group's GDPR policy) using Java, Spring Boot, PLSQL, Oracle, AWS RDS, and AWS ECS.- Implementation of procedures for calculating dynamic billing using PySpark jobs on Databricks.- Implementation of a PySpark job for transferring data from an Oracle database to S3 in Delta format.- Implementation of non-regression tests.
- EvidenceBData ConsultantEDUCATION AND E-LEARNINGJune 2021 - August 2021 (3 months)Development and maintenance of learning modules for teachers and their students (specific e-learning issues)- Development and maintenance of associated back-end microservices.- Application monitoring (Grafana)- Data cleansing and storage through parsing of a large volume of JSON code structures.
Recommendations
Be the first to recommend Malik
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- General Engineer DiplomaIMT Atlantique2019Traitement du signal et de l'information : - Traitement numérique du signal - Processus aléatoires - Traitement du signal - Technologies du multimédia - Analyse et optimisation - Pratique des signaux aléatoires et communications numériques - Modélisation et simulation stochastique Economie et sciences humaines : - conjoncture et politique économique - économie industrielle et stratégie d'entreprise - théorie de la décision - comportements des acteurs et structure des marchés - politique marketing - Management et politiques d'entreprises Informatique : - Génie logiciel et orienté objet - SI et bases de données Réseaux : - Réseaux IP - Réseaux mobiles et réseaux sans fil - Qos et ingénierie des réseaux Data science : - Implémentation du Système de Gestion de Données - Optimisation - Statistiques - Traitement d'image numérique - Management - Machine Learning et Systèmes Intelligents - Clouds
- Master 1 E3AUniversité Paris Sud XI (Paris Saclay University)2016Cette formation m'a fourni une base solide dans tous les domaines des sciences de l'ingénieur liés à l'électronique, l'énergie électrique, l'automatique, l'ingénierie informatique, les communications et le traitement du signal et de l'image.
Certifications
- Big Data Analysis with Scala and SparkCoursera - EPFL2020
- Machine LearningCoursera - Stanford2021