Sankar Biswas

I lead a team managing marketing data pipelines, bringing data from various sources into AWS, and constructing data marts to enhance data analysis capabilities. I played a crucial role in architecting the overall data strategy for the marketing data platform, utilizing Databricks, Spark, Python, SQL, Airflow, AWS EMR, Lambda, S3. Learning & creating POCs on LLM.

I led a team of engineers in architecting and building robust Spark ETL data pipelines on AWS for loyalty and audit data preparation. My role involves designing and developing scalable data architecture, ensuring efficient data processing using technologies like Databricks, Spark, Python, SQL, Airflow, AWS EMR, Lambda, and S3.

developed Spark streaming pipelines to process application logs, enabling real-time monitoring of application health. I primarily worked with Scala, Python, and SQL to deliver effective data processing solutions.