Freelancer profile translated to English.
Back to original languageAbout Omar
Senior Data Engineer, expert in Databricks, Spark and Delta Lake, passionate about pipeline automation and AI-assisted development (GitHub Copilot, Claude). Databricks certified, I create reliable and fast solutions to improve data quality and produce actionable insights.
French
Native or bilingual
English
Fluent
Can work on-site
Paris (up to 50km)
Experience
- AXASenior Data Engineer & AI-driven Development SpecialistOctober 2024 - Today (1 year and 8 months)Paris, FranceTeam: 1 Data Lead, 3 Data Engineers, 2 Business AnalystsProject: KYC Quality Monitoring and Control• Accelerated development and pipeline automation using AI (Claude, GitHub Copilot)• Built a centralized Golden Source and automated quality controls• Collaborated with business stakeholders to define specifications• Created unit tests and documentation• Developed and optimized Databricks and Spark pipelines• Optimized Spark jobs• Maintained pipelines in production and monitored performance• Visualized data quality using Power BI• Collaborated with business teams for remediation and reportingProject: Finance Distribution Reporting• Industrialized Business Values (BV) adhering to the Carbon framework• Collaborated with business stakeholders to define specifications• Created unit tests and documentation• Developed and optimized Databricks and Spark pipelines• Optimized Spark jobs• Maintained pipelines in production and monitored performance• Automated development and refactoring with GitHub Copilot• Generated automatic documentation• Generated specifications for existing applications via Copilot• Automated code review and deployment respecting the Carbon frameworkTechnical Environment: Azure Databricks, Unity Catalog, ADLS Gen2, Python, PySpark, Delta Lake, SQL, Git, Power BI, Claude AI, GitHub Copilot
- Société GénéraleData EngineerBANKING AND INSURANCENovember 2022 - September 2024 (1 year and 10 months)Paris, FranceTeam: Tech Lead, 4 Data Engineers, 3 Business Analysts, 1 Product Owner, 1 Scrum MasterProject: RWA Calculation Chain Transformation• Complete migration of Teradata pipelines to Databricks Lakehouse• Design, optimization, and partitioning of Spark pipelines for financial data• Implementation of Unity Catalog for data governance, security, and classification• Collaborated with business stakeholders to define specifications• Created unit tests and documentation• Optimized Spark jobs• Data quality control and reduction of rejections in the calculation engineTechnical Environment: Databricks, Spark, Delta Lake, AWS (S3, Lambda, Step Functions, Glue), Python, Scala, SQL, Jenkins, Git
- COLASData EngineerOctober 2020 - November 2022 (2 years and 1 month)Vélizy-Villacoublay, FranceTeam: 1 Data Lead, 2 Data Engineers, 2 Business AnalystsProject: Business Data Collection and Analysis Platform• Design of a Lakehouse architecture on Databricks• Development of Bronze/Silver/Gold pipelines with CDC management• Implementation of unit tests with ScalaTest and partial automation of flows• Collaborated with business stakeholders to define specifications• Created unit tests and documentation• Optimized Spark jobs• Collaborated with business teams for integration and reportingTechnical Environment: Azure Data Factory, Azure Databricks, Unity Catalog, Spark, Scala, SQL, Delta Lake, Azure DevOps, Git
Recommendations
Be the first to recommend Omar
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- Master 2: Information Processing and Data Analysis (TRIED)Université Paris Saclay2020Master 2: Traitement d'information et analyse de données (TRIED)
- Mohammedia School of Engineers2019