Vishaal M.

Data Engineering: Expert in designing, developing, and deploying scalable ETL pipelines leveraging AWS technologies including AWS Glue, Glue Jobs, Step Functions, CodePipeline, CodeBuild, S3, EC2, CloudFormation, CloudWatch, SNS, and Lambda functions. Proficient in data processing and transformation using PySpark within Glue environments and managing Glue Data Catalog tables for schema management and query optimization via Athena. Experienced in monitoring and alerting of data pipelines through CloudWatch to ensure reliability and prompt incident response. Strong hands-on experience implementing CI/CD workflows for data pipelines and infrastructure automation using GitHub and Bitbucket. Skilled in data ingestion, transformation, and loading from multiple sources such as APIs, databases, and files into target databases including DynamoDB, PostgreSQL, and Snowflake. Skilled in creating and managing AWS S3 buckets with efficient organization of datasets in multiple formats including Parquet, CSV, and JSON for optimized storage and query performance. Comprehensive expertise in building and managing end-to-end cloud infrastructure on AWS including EC2 instances, load balancers, VPCs, and networking components ensuring secure and robust deployment environments. Extensive experience developing RESTful APIs using Python and integrating them seamlessly with AWS services such as API Gateway and Lambda for serverless architecture solutions.

Integrated 3rd-party APIs such as Google Search Console, Google AdWords, Zoom, and Microsoft Teams (Webinars) to automate and streamline data retrieval and reporting processes across various platforms. Developed and maintained automated Python scripts for ETL, data processing, and system performance optimization, improving platform efficiency and reducing manual intervention. Designed and implemented end-to-end ETL processes on the AWS platform, facilitating secure, scalable, and reliable data movements between sources and target systems. Performed advanced web data API integrations to extract, normalize, and analyze key performance indicators, driving actionable business insights. Led the development and management of Tableau reports, from project planning to delivery, enabling stakeholders to visualize and interpret business-critical KPIs. Utilized PostgreSQL and Exasol databases for data transformation, aggregation, and backend automation, supporting real-time analytics and enhanced reporting capabilities. Process Innovation & Governance: Created automation applications and scripts to track and monitor data statistics, ensuring the integrity and accuracy of business data flows. Developed tools that reinforce process and platform governance, bolstering compliance and operational consistency across digital projects. Actively contributed innovative solutions and strategic ideas to support governance frameworks and digital agency operations. Championed the adoption of novel approaches in the PATSU Platform Project, fostering a culture of continuous improvement and technological innovation. Devised strategies to address and resolve complex technical requirements, leveraging automation and analytical technologies to deliver robust, scalable solutions.