Freelancer profile translated to English.
Back to original languageAbout Sandid
Hello,
Senior Data Engineer with over 6 years of experience in Big Data and Cloud. Expert in Spark, Scala, Pyspark, and CI/CD solutions. Specializing in data pipeline optimization, Vault modeling, and technical team support. Strong expertise in Azure and Databricks environments, with the ability to deliver high-performance solutions to meet complex business needs.
French
Fluent
English
Fluent
Can work on-site
Paris (up to 50km)
Experience
- BoursoramaData EngineerBANKING AND INSURANCEJune 2021 - Today (5 years)Paris, FranceProject Context:Development of real-time jobs and unit tests, implementation of business scenarios, and design and migration of business compliance scenarios.• Development of a real-time job to value Kafka data in Cassandra tables after processing and development of unit tests.• Development of a Spark job in Python to apply business rules and deploy it on Jenkins.• Design of tables and the solution to apply (reference tables, aggregation tables, etc.).• Understanding of business rules and the data sources to be used.• Development of a generic Spark job in Scala configurable for the 23 business scenarios.• Participation in the creation of a NIFI process to send generated alerts to Oxygéné.• Documentation of business rules and technical choices on Confluence.Technical Environment: Hadoop, Kafka, Spark, Scala, Pyspark, Hive, Cassandra, Nifi, Intellij (SBT), Jenkins.
- OrangeData EngineerTELECOMMUNICATIONSJanuary 2019 - June 2021 (2 years and 5 months)Project Context: Migration of an Oracle database and over 30 jobs and 100 sending reports to an Azure Big Data platform.Tasks Performed:- Development of data collection jobs (Processor Group) using Nifi, storing them in Data Lake Storage GEN2.- Processing of data streams by deploying a Spark Template job on Databricks (Scala).- Development of Spark Scala jobs on Databricks to migrate various sending reports from the old solution.- Optimization of Spark jobs to ensure optimal performance.- Execution of a testing phase for the various jobs using SQL queries, performing a comparative study with the old solution to verify data integrity.- Planning and scheduling of services with Data Factory.- Management of corrective and evolutionary maintenance operations and monitoring of various jobs through alerts and email notifications.Technical Environment: Azure, Spark Databricks, Scala, Cosmos, Data Factory, Nifi, Event Hub, Data Lake Storage Gen2.
- SofrecomData EngineerTELECOMMUNICATIONSJune 2018 - January 2019 (7 months)Goal: Implementing a predictive analysis system of customer behavior for contact centers during the call overflow period.- Development of jobs to collect data into the Datalake (Sqoop) from SqlServer to Hive.- Writing Hql scripts to prepare our dataset.- Data extraction and data visualization (statistical study) with Python, Jupyter.- Define the different customer classes at flexible contact centers and predict the number of calls to contact centers using Pyspark (Spark 2.4.0).- Present our results on Dashboard (PowerBi). Tools: Cloudera, Hadoop, Spark, SQLServer, Clustering (k-means, DBSCAN, hierarchical clustering...), regression (Decision trees, Random Forest, XGBOOST), Power BI
Recommendations
Be the first to recommend Sandid
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- PreparatoryPreparatory Institute for Engineering Studies of Monastir (IPEIM)2016Preparatory
- Engineering DiplomaSUP'COM2018Diplôme d'ingénieur en Big Data