Ali Ahmed Hadid

Mission: Implement the various components of the aeronautics client's big data platform (fleets, weather, incidents, positions)

Tasks:

Design and implement platform components: data ingestion, processing, normalization, and exposure.

Improve the common framework (monitoring, CI/CD, API, testing, performance, code quality…)

Ensure compliance with Group standards and collaborate with stakeholders.

Improvement of the common framework (monitoring, CI/CD, testing, performance, resilience, development practices, and code quality)

Environment: VS Code, Docker, Kubernetes, Airflow, AWS

Languages: Python, Scala

Technologies: Spark, GitLab, CI/CD, MongoDB, CloudWatch, SparkUI

Lambda, API-Gateway, S3, Jira

Systems: Windows, Linux, AWS

Missions:

Contribute to the implementation of the various components of the client's data platform.

Achievements:

Set up a data platform based on Gitlab and MongoDB for versioning data scientist datasets

•

Development of various FastAPI microservices

•

Maturation of Python code from machine learning Python notebooks

•

Demonstrations and POCs of newly developed features

•

Analysis and execution of various unit/integration tests for project components

.

Technical documentation with Sphinx

Mission: Analysis and development as well as support of certain applications used within Cora hypermarkets, mainly on AWS big data environments. Tasks:

• The implementation of a data platform dedicated to the customer loyalty program

• Collection of multi-source data (parquet, csv) to feed the DataLake S3

• Realization of developments, unit tests and technical recipes (Python, Spark)

• Exposing and testing the results of spark processing for the business in the Snowflake environment (Sql)

• Optimization of marketing costs and sms sent to customers

• The processing of multi-source data from the DataLake (parquet, csv)

• Configuration of parameters and calls to python functions (Yaml).

• Digitization of receipts

• Deployment and testing of developments in the AWS environment

• Participation in various agility meetings (Jira, Daily) Technical environment :

• Environment: AWS (S3, EMR, Pipeline), PyCharm, Snowflake

• Languages: Python, Sql

• Technologies: Spark, git, Jira

• Systems: Windows, AWS Ec2