Freelancer profile translated to English.

Description

AWS & Multi-Cloud Data Platform Expert with Proven Track Record

I specialize in AWS architectures, cloud migrations, and data platform modernization with 15+ years of experience in enterprise projects. Experienced in Enterprise Architecture Advisory, Technical Reviews, and Team Mentoring.

Current Focus Areas:

AWS Services: Lambda, Glue Streaming (Kafka Consumers), Glue ETL, API Gateway, Aurora RDS, S3, DMS, Athena, Event Bridge
AWS Databricks, Azure Databricks, PySpark
Power BI Table Performance Optimization & MicroStrategy 10/11
Redshift RA3 (Distribution Keys, Sort Strategies, WLM Queues)
Unity Catalog, Delta Lake, Workflow Orchestration
Infrastructure as Code (Terraform/Terraspace, Liquibase)

Proven Track Record:

Appian: Enterprise Data Platform with AWS Glue & Lambda, Apache TIKA Integration
Circle K: 6M+ Daily Transactions, AWS Cost Reduction, 600+ Locations
Couche-Tard: Power BI + Redshift Performance Boost, 3-5x Photon Acceleration
TotalEnergies: AWS Transfer Family, SAP BOXI SFTP (Fingerprint) Integration, Process Automation

AWS Expertise:

Compute: Lambda, ECS (Docker Container), EC2
Data: Glue, Redshift, Aurora RDS, DMS, PostgreSQL, Athena
Storage: S3 (Lifecycle Policies, Cross-Account Access)
Integration: API Gateway, Transfer Family, EventBridge, GitHub OIDC
Security: IAM, KMS, OIDC, SCIM Passthrough

Data Engineering & Governance:

Python Development: Lambda, PySpark (Databricks), Glue Jobs, Shell/SFTP Scripting (SQL, Apache TIKA)
Data Processing: ELWIS, PEGELONLINE WSV, DWD Datasets
Compliance: GDPR Implementation, Data Classification Strategies
Documentation: Comprehensive Standards, API Design Patterns
Performance: Distribution Keys, Sort Strategies, WLM Queue Optimization

Additional Core Competencies:

Database Expertise: Aurora RDS Postgre, Oracle (OCP 11g DBA certified), Redshift, Exasol
Data Modeling, DV2.0, 3NF, Canonical

Industry field of expertise

Languages

German
Conversational
English
Native or bilingual
German
Fluent
English
Fluent

Workplace preferences

Remote only

Primarily works remotely

Transport - Public Sektor
Lead Data Engineer - Appian Data Platform
TRANSPORTATION
November 2024 - Today (1 year and 7 months)
Frankfurt am Main, Germany
Leading enterprise data platform design and implementation with scalable architecture patterns and compliance-driven development.

Enterprise Architecture & Solution Design
Designed Medallion Architecture (Bronze/Silver/Gold) supporting multiple data domains
Implemented serverless ETL pipelines using AWS Glue and Lambda with PySpark DataFrames
Created architectural blueprints for event-driven data ingestion using API Gateway and Lambda
Built Terraform modules following modular design patterns for reusable infrastructure
Real-Time Kafka Streaming Pipeline
Architected enterprise Kafka-to-PostgreSQL streaming ETL processing 10,000+ messages/hour
Engineered AWS Glue Streaming job with advanced Kafka consumer group management
Built production monitoring: CloudWatch metrics, S3-based DLQ, correlation IDs
Implemented dual-layer database resilience: psycopg2 + RDS Proxy achieving 95%+ success rates
Designed micro-batch architecture with configurable windows
Public Transport Infrastructure - Real-Time Search
Architected OpenSearch CDC ingestion processing 10,000+ records/minute
Implemented production Lambda function (Python 3.13, 3GB) with 1,000 events/batch
Resolved security compliance issues (tfsec, tflint, Bandit, Semgrep)
Achieved zero-downtime deployment using versioned OpenSearch Pipeline v2
Infrastructure Patterns & Best Practices
Standardized Terraform module structure across 15+ modules
Implemented GitOps workflow with automated security scanning
Applied patterns: conditional resources, lifecycle controls, security group reuse
Established CloudWatch monitoring for pipeline health metrics
Outcome:Enabled scalable platform supporting multiple domains with real-time streaming, search functionality, and standardized patterns ensuring compliance and operational excellence

Technologies:AWS (Lambda, Glue, S3, API Gateway, Aurora Postgre, OpenSearch, RDS Proxy), Kafka, PySpark, Terraform, psycopg2
Terraform AWS S3 AWS Lambda OpenSearch API Gateway
Circle K
Project Data Manager / Data Engineer
ENERGY AND UTILITIES
April 2025 - July 2025 (4 months)
Berlin, Germany
Architected and executed AWS-to-multicloud migration for Circle K's data platform (AWS/Azure Databricks), designing cross-cloud data transfer solutions that modernized infrastructure while maintaining zero-downtime operations for 600+ retail locations.

Cross-Cloud Migration Architecture:
S3-to-Azure Blob pipeline via AWS Databricks for cross-cloud data transfer from legacy AWS to Circle K Azure
Phased migration strategy maintaining single legacy AWS account for synchronization while transitioning workloads to Circle K environments
Unity Catalog volumes collaboration with Circle K metastore admins, defining schema structures and access patterns
Migrated from low-level Spark RDDs to high-level DataFrame APIs, enabling Photon acceleration for performance gains

Infrastructure as Code & Automation:
Implemented Terraspace/Terraform Redshift stack and modules, with RA3 node configuration
Implemented Redshift native schedules with associated IAM roles for automated ETL workflows
Managed database scripts for database and schema definitions / user access management

Data Pipeline Modernization:
Refactored multiple Python notebooks from AWS Databricks to Azure Databricks, optimizing for Photon accelerator
Implemented Unity Catalog Delta tables for daily POS data processing with automatic schema evolution
Created weekly KPI / reports using Databricks views analyzing product volumes and sales metrics
Migrated data pipelines from sequential processing to parallel Spark operations

Outcome:orchestrated zero-downtime migration serving 6M+ daily transactions; scalable cross-cloud architecture supporting both AWS and Azure workloads, reduced operational costs by 40% through infrastructure consolidation.

Technologies: Azure Databricks, AWS Databricks, Terraform, Unity Catalog, Delta Lake, Redshift (RA3), S3 Cross-Account Access, IAM AssumeRole, Photon Engine, PySpark, Azure Blob Storage
Databricks AWS S3 Python PySpark
Couche-Tard Deutschland GmbH & Co. KG
Project Data Manager
ENERGY AND UTILITIES
November 2023 - March 2025 (1 year and 5 months)
Berlin, Germany
Managed cloud data infrastructure, focusing on Redshift optimization, Unity Catalog implementation, and AWS Databricks. Led performance tuning initiatives reducing query execution times across Power BI workloads.

Redshift Performance Engineering:
Implemented distribution keys, sort/interleaved keys based on query pattern analysis
Established Workload Management (WLM) queues with memory allocation optimization for concurrent Power BI refresh jobs and Pipeline workloads
Configured RA3 node clusters with managed storage enabling independent compute/storage scaling
Created monitoring views for table statistics, query performance, and skewness tracking
Advised VACUUM strategies during pipeline runs, optimizing table performance
Implemented automatic snapshot schedules for recovery scenarios

Databricks Platform Establishment:
Secured AWS Databricks access from parent organization through governance approval process
Implemented Terraform modules for Unity Catalog setup with three-level namespace (catalog.schema.table)
POCs for AWS Databricks adoption over Redshift, for performance and cost improvements, using Delta tables
Built data pipelines for public and internal data sources, using Spark RDD parallel processing
Built data quality monitoring completeness KPIs

AWS Infrastructure Management:
Managed subnet groups and CIDR block allocations for Redshift cluster isolation
Configured cluster resize operations from dc2.large to RA3.xlplus nodes with minimal downtime
Managed Power BI Gateway Data Sources, and Network Firewall Whitelisting b/w OnPrem and Cloud using FQDNs

BOXI to Cloud Data Integration:
Integrated SAP BOXI SFTP Exports with AWS Cloud, using AWS Transfer Family and Fingerprint calculation approach

Stack:AWS Redshift (RA3), Databricks, Unity Catalog, Terraform, WLM, Distribution Keys, Interleaved Sort Keys, Power BI, S3, Spark RDD, Python, Delta Lake
Databricks AWS S3

Check out Muhammad Moiz's experience

Reyhan

Couche-Tard Deutschland GmbH & Co. KG

Reviewed on 10/10/2024

Data Management: Mohammad MOIZ AHMED has demonstrated outstanding skills in the area of data management. He has efficiently organised and updated our data management systems, resulting in a significant improvement in data integrity and accessibility. His structured and methodical approach has enabled us to optimise data management and accelerate processes. Cloud Administration (AWS): Mohammad MOIZ AHMED demonstrated a strong understanding and technical expertise in cloud administration. He implemented effective strategies to secure and manage our cloud infrastructure. As a result of his efforts, system downtime was minimised and overall operational efficiency increased. Communication and Teamwork: Mohammad MOIZ AHMED has proven to be an excellent communicator, working effectively with internal teams and external partners. His ability to communicate complex technical information clearly and concisely has greatly improved knowledge sharing and collaboration within the team. Problem Solving: Mohammad MOIZ AHMED demonstrated exceptional problem-solving skills. He proactively identified potential pain points and developed innovative solutions to address them. His analytical thinking and proactive approach helped to overcome technical challenges quickly and efficiently.