Jonathan Awo

Within STIME, my role is to process the various data flows (Stocks, Sales, Orders, Prices, Promotions, Suppliers, Invoices, etc.) from the Home Equipment entities (Bricorama, Bricomarché, and Bricocash) to enable points of sale and finance to monitor their activities.

➢ Migration of old SQL Server stored procedures to the Databricks Azure platform

➢ Implementation and refactoring of jobs (spark/scala, pyspark) with Databricks for processing 30 daily flows (Purchases, Suppliers, Stocks, Purchase Prices, Cost Prices…)

➢ Optimization of DeltaLake and SQL Server tables

➢ Creation of the pipeline with Azure Data Factory

➢ Data storage on Databricks DeltaLake

➢ Documentation of developments

➢ Deployment of releases via Azure Devops

➢ Support for the team on agile methodology best practices

Within the investment and financing bank, my role consists of processing financial flows and making financial reports available to business teams.

➢ Migration of Java batch jobs to Spark Scala

➢ Study and implementation of data pipelines (Volume ~30 million operations/day)

➢ Payment methods mapping

➢ Payment anomaly detection / Anti-Fraud

➢ Extraction of banking operations

➢ ETL job scheduling (Oozie, Crontab)

➢ CI/CD for automatic deployment of jobs with Ansible and Jenkins

On behalf of the publisher DataOps Saagie, I am working with one of its clients as a Data Engineer. My mission is to implement data processing pipelines and provide support on the Saagie platform: SaagieManager.

➢ Participation in technological choices for the implementation of the Big Data architecture

➢ Configuration of a filebeats yaml file for sending log files to Kafka

➢ Implementation of a Scala Akka Stream job for sending log files to Kafka

➢ Creation and configuration of Kafka topics

➢ Creation of Elasticsearch and Kibana docker images

➢ Real-time job implementation (Structured Streaming) spark/scala for reading from Kafka, cleaning logs, and writing to Elasticsearch

➢ Deployment of spark jobs on AWS EMR and on the Saagie Manager dataops tool

➢ Implementation of Kibana dashboards for Vallourec's top management

➢ Batch job implementation pyspark for processing tube data stored in hdfs/S3 and storing data in Hive and Impala tables

➢ Implementation of a calculation job to measure the degree of compatibility between different tubes.

➢ Unit tests with ScalaTest, JUnit, and UnitTest

➢ Setup of a CI chain with gitlab and Docker