Yassin E.

RUBI PROJECT: Development and RUN of a platform for data ingestion, analysis, aggregation, and transcription, forming an enterprise data catalog and data visualization, on the Azure cloud.

AREA OF INTERVENTION:

● Analysis of needs and existing architecture, composed of SSIS/SSAS and SharePoint sites

● Design of architecture diagrams and a migration plan to the Azure cloud, composed of several milestones.

● Implementation of the first milestone, composed of the architectural foundation, deployed via Terraform

● Development of a library of Terraform modules and deployment pipelines via Azure DevOps to automate infrastructure as code.

● Deployment of three isolated environments on three Azure subscriptions, as well as the beginning of decommissioning on-premise resources.

● Configuration of Azure Data Factory, acting as ETL, and Azure Databricks, serving as the execution environment for Python/PySpark jobs.

● Implementation of CI/CD pipelines allowing packaging and deployment between environments of developments on Azure Data Factory as well as the Python package, deployed on the Databricks file system.

● Configuration of the enterprise scheduler, OPCON, used to run Azure Data Factory jobs daily, including the creation of OPCON schedules, OPCON jobs, and Python scripts.

● Implementation of monitoring for the RUBI platform, via OPCON alerts calling SCOM, as well as Azure Monitor alerts, also intercepted by SCOM.

● Implementation of on-premise virtual machines, serving as data gateways for Power BI, which source from Azure Databricks via private links, in order to exclusively use private flows and integration runtimes for Azure Data Factory.

● Implementation of automation for Power BI report deployments, including the deployment of reports/datasets/apps, the creation of data sources for data gateways, and the creation of workspaces with business rules.

● Progressive commissioning of different Power BI reports, in different environments (Dev/Pre-Production/Production)

● Run of the project, with incident management and progressive improvement of the platform

TECHNICAL ENVIRONMENT:

● Azure Cloud (Data Factory, Databricks, Private DNS/Link/Endpoint, ADLS v2, Key Vault)

● Python (PySpark/Pandas) - PowerShell

● Terraform/Ansible

● Microsoft Power BI

● Azure DevOps

Datalab PROJECT: Monitoring and maintenance of a data scraping, processing, and delivery platform via a ReactJS Web application.

AREA OF INTERVENTION:

● Analysis of the existing product architecture, as well as the application's various pain points

● Redesign of the deployment of various resources on Azure via Terraform, thus automating the construction/destruction of the application

● Implementation of architecture diagrams to document the application, with the aim of easier maintenance and improvement

● Deployment of several redundant and isolated environments, while closing all network openings, and routing communications via Privatelink/Fortinet

● Maintenance and run of the AKS cluster, including incident management, as well as the deployment of new resources

● Development of Helm charts for deploying new resources.

● Development of Argo workflow templates for new data scraping.

● Maintenance and improvement of the project's CI/CD pipelines, for deploying various wheel packages (Python)

● Improvement of the Prometheus/Grafana monitoring platform

● Run of the ReactJS application, including assisting developers with incidents/bugs

TECHNICAL ENVIRONMENT:

● Azure Cloud (AKS, Private DNS/Link/Endpoint, ADLS v2, Key Vault)

● Argo Workflow/Helm

● Python (PySpark/Pandas)

● Terraform

● Azure DevOps

FinOps Analysis PROJECT: Creation of a system for analyzing financial data from all group subsidiaries, across all used clouds, and visualizing this data through Power BI reports.

 Analysis of financial data volumes, various business issues (e.g., shared ASEs by different entities), and different cloud providers to monitor (Azure, AWS, Openshift)

 Development of Azure Data Factory pipelines ingesting raw data from an Azure API, making it available first in blob storage, then in Azure Cosmos DB

 Development of Azure Databricks notebooks in PySpark/Pandas ingesting large volumes (hundreds of thousands of lines per processing) of billing data (from Blob/Cosmos DB), processing them in an optimized manner to analyze and redistribute each cost, finally storing them in Cosmos DB or as blobs

 Implementation of monitoring and alerting for Data Factory pipelines and Databricks notebooks to ensure 99% availability through runbooks and native tools

 Onboarding of AWS and Openshift cloud providers, with transfer of billing data from these cloud providers to an ADLS v2, and creation of specific Databricks notebooks for each cloud provider, managing different business rules and client constraints

 Creation of Power BI dashboards for visualizing billing data

 Creation and maintenance of a deployment system for the entire solution (Data Factory, Runbooks, Databricks, Monitoring, Blob/Table Storage, Cosmos DB) automated via Azure DevOps release pipelines

 Creation of Terraform modules for deploying Data Factory/Databricks infrastructure to client entities

 Onboarding of different entities by collecting requirements, creating notebooks and pipelines, and the specific Power BI dashboard

DevTestLabs PROJECT: Automation of managed development environment creation on the Azure cloud using the DevTestLabs service.

 Creation of custom images (golden images), serving as a base for creating managed virtual machines, via Azure DevOps release pipelines, and ARM templates taking software via artifacts

 Creation of Azure DevOps release pipelines to deploy virtual machines using custom images on the DevTestLabs service, aiming to deploy a secure, automated, and scalable development environment

 Creation of Terraform modules for deploying target resource groups containing DevTestLab infrastructure

 Automation of resource deployment using Terraform scripts via Github Actions

 Creation of release pipelines automating the deployment of various services used for the solution (ADLS, Key Vault, DevTestLabs, Virtual Network, Firewall)

 Study of each entity's needs to perform necessary network configurations on Azure Firewall, network peerings, and traffic openings when necessary

 Creation of an automated destruction and reconstruction system via Azure DevOps and Azure Functions to destroy each environment every two months to minimize security risks and patch virtual machine images and rebuild them automatically

Active Directory Management PROJECT: Automation of Azure Active Directory management through the creation of Azure Functions to enable AXA entities to be autonomous in their actions without granting them direct rights on Active Directory.

 Analysis of the different needs of entities with actions to perform on Active Directory

 Development of generic libraries in PowerShell to automatically create objects within Active Directory and create RBAC

 Deployment of a secure environment to consume Azure Functions from restricted network flows via either an Application Gateway with incoming flow whitelisting, or an App Service Environment supported by an Azure Firewall, depending on the entity's needs and budget

 Automation of resource group creation and various resources (Azure Functions, Application Gateway, App Service Plan, Blob Storage, Key Vault) as well as Azure Functions deployment via Azure DevOps release pipelines

Tenant Monitoring PROJECT: Development and deployment of a web app for monitoring the migration of virtual machines from IBM private cloud/on-premise to Azure/AWS

 Development of a ReactJS web app displaying cloud-to-cloud migration data, stored by runbooks on table storage

 Automation of web app deployment via Azure DevOps on an App Service

 Creation of release pipelines automating the deployment of various services used for the solution (ADLS, Key Vault, DevTestLabs, Virtual Network, Firewall)

 Iterative addition of use cases and additional data stored on Cosmos DB

Data Migration PROJECT: Migration of data from either Azure or an external source to Azure.

 Study of data volume, sensitivity, and transfer frequency

 Automation of virtual machine, ADLS v2, and network peering creation so that data can transit from origin to destination via private flows using Terraform modules

 Creation of scripts using Az Copy for Azure-to-Azure data import and export

 Data migration using Azure Migrate for External-to-Azure transfer cases

AWS Infra Automation PROJECT: Automation of infrastructure deployment and provisioning on AWS for AXA client entities.

 Analysis of the different needs of entities on AWS

 Development of Terraform scripts to create IaC (Infrastructure as Code), creating multiple services (EC2, VPC, DynamoDB, S3, Lambda)

 Development of Ansible playbooks to provision and configure EC2 resources on AWS

 Development of Jenkins scripts to provision and configure EC2 resources on AWS

AWS Tagging Monitoring PROJECT: Development of AWS resource discovery scripts to populate a ServiceNow CMDB to verify the tagging level of resources.

 Development of Python scripts for AWS resource discovery to verify tag compliance, stored in a DynamoDB database

 Sending email notifications via AWS SNS to affected teams

 Sending information to a ServiceNow CMDB via a REST API

AWS IAM Management PROJECT: Development of IAM configuration files to apply a "least-privilege" practice on different AWS accounts.

 Development of IAM role configuration YAML files

 Integration of Github IAM configuration deployment on AWS via CodePipeline