Freelancer profile translated to English.
Back to original languageAbout Andre
With 6 years of experience, I have acquired solid expertise in data management and analysis. I worked for 3 years as a relational database architect, designing and optimizing data models. I then transitioned to the role of Data Engineer for 3 years, where I built large-scale data pipelines.
In parallel, I participated in Data Science projects, applying my skills in statistical modeling and machine learning. My client experience has allowed me to develop skills in communication and project management.
Passionate about data, I am ready to take on new challenges and bring added value to your organization.
French
Native or bilingual
English
Fluent
Can work on-site
Paris (up to 50km)
Experience
- AtosData EngineerTECHDecember 2023 - January 2025 (1 year and 1 month)Bezons, FranceMission Objectives at SANOFI
- Deploy a CI/CD pipeline to automate governance and monitoring.
- Automate access management and secure data usage.
- Implement archiving and deletion pipelines to optimize storage.
- Ensure monitoring and debugging of data pipelines.
- Creation and optimization of transformation pipelines on Snowflake
Accomplishments1. Data Access and Governance- Identified user needs and defined roles via AWS IAM.
- Automated permission deployment with CloudFormation and GitHub Actions.
- Implemented simplified and centralized access management.
2. Creation of Archiving and Deletion Pipelines- Developed Lambda functions to move and delete obsolete files.
- Automated the process with AWS EventBridge to execute tasks periodically.
- Used CloudWatch to track executions and detect errors.
3. Pipeline Monitoring and Debugging- Monitored file extraction rates with Splunk.
- Analyzed logs on CloudWatch to detect and classify errors.
- Collaborated with Data teams to optimize processes.
4. Data Loading & Pipeline Automation (S3 → Snowflake)- Loaded data from S3 to Snowflake via external stages with automatic new file detection
- Orchestrated transformation tasks triggered after CDC activation
Technical Environment- Languages: Python, PySpark, SQL, YAML
- Cloud & Data Services: AWS (IAM, S3, Lambda, Glue, Athena, CloudFormation, Step Functions, EventBridge)
- Snowflake: Snowsite, Snowpipe, Snow Task, Snow SQL
- CI/CD & Monitoring: GitHub Actions, Splunk, CloudWatch, Terraform, Docker
- AtosData EngineerTECHOctober 2022 - September 2023 (11 months)Bezons, FranceMission Objectives
- Automate data migration by managing different formats and encodings.
- Implement a reconciliation system to verify consistency between old and new data.
- Ensure data integrity by guaranteeing the quality and security of transfers.
Accomplishments1. Migration Pipeline Automation- Developed Python scripts for automatic splitting of large files.
- Converted and standardized encodings to UTF-8 to ensure compatibility.
- Implemented a data ingestion pipeline into a MySQL database.
2. Database Optimization and Security- Defined a robust architecture with referential integrity and validation constraints.
- Implemented an access and permissions management system.
- Created dynamic views to limit exposure of sensitive data.
3. Orchestration and Automation with Airflow- Designed and deployed an Airflow pipeline for periodic retrieval of files from AWS S3.
- Implemented automated quality checks (formats, integrity) and notifications in case of anomalies.
4. Data Analytics (BigQuery)- Prepared and exposed data to an analytical Data Warehouse (BigQuery) for reporting and BI usage
- Wrote analytical SQL queries (views, aggregations) for business data consumption
- Modeled analytical data and structured it for BI (tables, views, indicators)
Technical EnvironmentLanguages: Python, SQLDatabases: MySQLTools and Frameworks: Web2Py (MVC), GitHub, Pandas, Airflow, AWS S3, Big Query - Horiba MedicalData EngineerTECHMay 2021 - October 2021 (5 months)Montpellier, FranceMission Objectives
- Automate the collection and processing of data from hematological analyses.
- Implement a robust database for storing and analyzing blood cells.
- Develop a user interface for visualization and manual segmentation.
- Optimize data quality by applying integrity and validation rules.
Accomplishments1. XML Data Extraction and Processing- Developed Python scripts to parse and structure data from thousands of XML files.
- Cleaned and standardized data to ensure its use in a relational database.
- Implemented automatic file verification to detect anomalies.
2. Database Design and Optimization- Compared MongoDB and PostgreSQL to determine the best storage solution.
- Implemented PostgreSQL with an optimized relational model for cell segmentation.
- Defined integrity constraints and validation rules for biological data.
3. User Interface Development- Designed and developed the front-end and back-end with PyQt5 to allow biologists to visualize and annotate blood cells.
- Implemented a fluid interaction system between the database and the application.
- User testing to optimize the ergonomics and performance of the interface.
4. Documentation and Deployment- Wrote clear technical documentation to facilitate project maintenance and evolution.
- Implemented unit and functional tests to ensure application reliability.
- Deployed the application on Horiba Medical's intranet.
5. Technical Environment- Languages: Python, SQL
- Databases: PostgreSQL
- Frameworks & Tools: PyQt5, Sphinx, GitLab
Recommendations
Be the first to recommend Andre
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- Specialized Master in Big DataENSIMAG - Grenoble2022Au cours la formation de 2 ans nous avons été formés en Software Engineering, Machine learning, Data Engineering, Big Data, Maths, Data Mining, Data Visualisation, Statistiques et Probabilités
- Master 2 in Computer ScienceUniversity of Yaoundé 12012Au cours de la formation de 2 ans nous avons été formés en Software Engineering, MySQL and PostGreSQL Data Base Architecture, Programmation HTML, CSS, SQL, C et PHP, Configuration de réseaux locaux (LAN)
Certifications
- Databricks Certified Data Engineer AssociateDatabricks
- Google Cloud Associate Cloud EngineerGoogle