About Pierre-Emmanuel
English
Native or bilingual
French
Native or bilingual
Experience
- Wealth Wizards LtdData Platform EngineerAugust 2022 - January 2024 (1 year and 5 months)Royal Leamington Spa, United KingdomAs a Data Platform Engineer, I was part of the data team responsible for solidifying the data lake in S3 and improving the data pipelines initially built with AWS Glue for data processing, AWS Kinesis for ingestion, and AWS Athena for querying.Key Contributions:
- Set up Power BI to access data via AWS Athena, enabling the creation of insightful data dashboards.
- Upgraded the original data pipeline from Glue jobs to Apache Spark (PySpark) for improved robustness and testability, still managed by Glue.
- Built a full delivery pipeline in GitLab using Pulumi (Python) for infrastructure as code.
- Introduced Delta Lake tables once AWS Athena released better support, primarily for merging and compaction capabilities.
- Created a Node.js module used by our Delivery Teams to send data to Kinesis for ingestion in the data lake.
- Wrote Spark jobs in PySpark and Lambda functions in Python for data processing.
- Led the migration from Power BI to AWS QuickSight, setting up the necessary infrastructure and authentication configurations to access Athena data protected by KMS.
- Worked with data scientists to develop the first iteration of a SageMaker pipeline for training and serving machine learning models.
- Managed all infrastructure for S3, Kinesis, Athena, and QuickSight using Terraform, while Glue/Spark jobs were orchestrated via Pulumi in Python.
Technologies Used:- Languages: Python, PySpark, Node.js
- CI/CD: GitLab, Bitbucket, GitHub
- Data & Cloud: S3, Kinesis, IAM, Glue, Athena, KMS, Lambda, SageMaker, Delta Lake
- Infrastructure as Code: Terraform, Pulumi (Python)
- Wealth Wizards LtdSenior Platform EngineerNovember 2017 - August 2022 (4 years and 9 months)Royal Leamington Spa, United KingdomAs a Senior Platform Engineer, I played a key role in shaping the platform's infrastructure, deployment pipelines, and overall developer experience. Working closely with both the Platform and Delivery teams, I focused on enhancing deployment processes and empowering engineers with self-service capabilities.Key Contributions:
- Enhancements to the existing micro-services architecture running on Kubernetes in AWS and Docker based CI/CD pipeline of Node.js, ExpressJS services backed by MongoDB clusters
- Upgraded the Jenkins cluster by migrating the master instance to Kubernetes and integrating Git-based config management.
- Designed a new delivery pipeline using the Serverless Framework, ensuring seamless AWS deployments while maintaining Kubernetes compatibility.
- Strengthened AWS security by implementing KMS encryption, backups, and cross-region data replication.
- Automated DynamoDB table replication via Kinesis streams and developed a Node.js module for automatic HTTP header propagation in test environments.
- Managed the AWS infrastructure with Terraform, creating scalable, reusable Terraform modules.
Tech Stack:- Infrastructure: Kubernetes, Docker, MongoDB, ELK
- CI/CD: Jenkins, Bitbucket, GitHub, Bash, Groovy
- Serverless: AWS Lambda, API Gateway, DynamoDB, S3, Cognito
- Cloud & Security: AWS (EC2, IAM, KMS, VPC, Route53, etc.), Terraform, Vault, Consul, Istio
- Workflow: Jira, Confluence
- Rare LtdBuild And Release EngineerApril 2016 - April 2017 (1 year)Atherstone, United KingdomAs a member of the Release Pipeline team at Rare, I was responsible for managing the deployment process for their latest Xbox title. My work focused on improving the reliability and scalability of the build farm, automating deployment and provisioning, and gathering data for telemetry and analyticsBuild Pipeline & Automation:
- Worked with a mix of C++, C#, and third-party tools, including Perforce, MSBuild, Visual Studio, TeamCity, and Octopus Deploy.
- Administered and supported the TeamCity Build Farm, which included physical agents, virtual machines (Windows Server), and Xbox devkits.
- Developed and configured TeamCity plugins in Java/Groovy.
- Automated TeamCity upgrades using PowerShell DSC.
- Managed data retention strategies for handling daily terabytes of build artifacts.
Infrastructure Provisioning:- Created ARM templates to provision a fully functional Service Fabric cluster on Azure.
- Provisioned a RabbitMQ cluster on Azure using Linux (Ubuntu).
- Managed cluster template versions via TeamCity and Octopus Deploy.
Telemetry & Analytics:- Assisted in deploying a Splunk cluster for build farm telemetry.
- Automated Splunk component provisioning with PowerShell DSC.
- Developed custom Splunk apps in C#/PowerShell to collect data from the TeamCity REST API.
- Designed Splunk reports, dashboards, and alerts for actionable insights.
Performance Monitoring:- Set up and administered a 3-node Elasticsearch cluster with a Kibana frontend for performance testing stats.
- Migrated the Elasticsearch cluster and data from Azure to on-premises virtual machines.
- Automated provisioning using PowerShell DSC.
Recommendations
Be the first to recommend Pierre-Emmanuel
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- CNAM2012CNAM
- DUT, Informatique IndustrielUniversity of Bourg-en-Bresse2000DUT, Informatique Industriel