Léonce E.

Data Engineering & Distributed Pipelines

Design of batch and streaming pipelines on Spark, Kafka, AWS EMR, orchestrated with Airflow or Dagster, versioned and deployed via GitOps.

Monitoring of performance, latency, and throughput in production with integrated tests (unit, contract, quality).

Infrastructure as Code & Data-Oriented DevOps

Automated provisioning via Terraform, Ansible, containerization with Docker, and CI/CD via GitHub Actions or GitLab CI.

Continuous integration of transformation pipelines (dbt, PySpark) with structured versioning.

Data-Centric Monitoring & Observability

Complete Prometheus + Grafana + ELK stack to monitor the health of pipelines, jobs, and data volumes.

Integration of anomaly detection rules, intelligent alerts, and automated remediation workflows.

Data Quality, SLOs, and Governance

Implementation of quality controls (dbt tests, Python assertions, Soda SQL), data SLOs, dashboards for business teams.

Contribution to the documentation of flows, schemas, lineage, and data catalogs.

🚀 Examples of achieved impact:

🔻 Reduction of MTTR by 60% on data incidents via intelligent monitoring and predictive alerting

📊 Processing of 1.5 million events/second on a gray market surveillance platform

⚙️ Reduction of reporting generation time from 6 hours to 20 minutes via Python automation + Tableau API

🧠 Implementation of an ML model to predict risk areas (counterfeiting, supply chain disruptions)

📊 Gray Market Surveillance Platform – Real-time Multi-Industry Processing

→ Design and deployment of a real-time monitoring solution for the parallel market, integrating fragmented and heterogeneous data sources.

→ Processing of over 1.5 million transactions/second via Kafka and Spark Streaming, correlating price anomalies, suspicious volumes, and weak signals on electronic, pharmaceutical, and consumer products.

→ Creation of dynamic dashboards in Tableau to provide clients with a clear and actionable view of sector-specific risks.

🔍 Predictive Risk and Opportunity Framework – Machine Learning & Proactive Alerting

→ Design of a predictive analysis model based on historical trade data and machine learning to detect risk areas, counterfeiting trends, and supply chain vulnerabilities.

→ Secure pipelines on AWS EMR and Redshift, incorporating data security and confidentiality constraints.

📈 Automation of Client Reporting – Personalized Data as a Service

→ Automation of personalized report generation in Python, integrated with the Tableau API, allowing clients to access self-service insights.

→ Reduction of reporting times from several hours to a few minutes, facilitating instant and contextualized decision-making.

⚙️ Technical Environment

☁️ Cloud: AWS (EMR, S3, Redshift, IAM)

🧪 Languages: Python, SQL, Java

⚙️ Tools & Infra: Kafka, Spark Streaming, Terraform, Ansible, Jenkins, Tableau

As a data-focused software engineer, I designed, optimized, and maintained data-intensive backend systems through an international freelancing platform. My role covered data modeling, real-time data extraction, and the reliability of critical pipelines for business decisions.

🔧 Main Responsibilities:

Development of robust data pipelines for user performance tracking, reputation systems, and freelancer scoring.

Implementation of data warehousing solutions with PostgreSQL and Redshift for marketing and product teams.

Refactoring of the backend (Node.js & Python) to separate business and data responsibilities (Clean Architecture).

Integration of unit and end-to-end tests to ensure the integrity of critical data processing.

Collaboration with product, marketing, and support teams to translate business needs into structured backend logic.

🚀 Notable Results:

📉 Reduction of report generation time from 6 hours to 20 minutes.

🔄 Implementation of an automated ETL pipeline with Airflow and SQL on AWS, improving data freshness by +85%.

🛠️ Participation in the migration of a monolith to a service-oriented architecture (Node.js microservices).

🧠 Improvement of internal decision-making tools through metrics on freelancer quality, used to feed the ranking algorithm.