Usama Shah

Context:

Implement solutions to help data scientists deploy distributed ML models in production.

Responsibilities:

- Development of elegant and simple code for high-performance, reliable, and maintainable architectures and applications.

- Development, packaging, and deployment of the internal distributed AutoML solution with Spark/H2O.aio

- Design and development of a PySpark template that automates dependency packaging and deployment of Spark jobs on isolated HDP platforms with (Shell/PEX (cluster pack)/Gitlab CI/CD)

- Setup and maintenance of CI/CD pipelines (Gitlab CI)

- Setup of continuous, automated testing (unit, integration, non-regression, and load) and coverage.

- Migration of Spark 2.2 -> 2.4, HDP -> CDP production assets

- Debugging and optimization of Spark jobs in HDP

- Development in an agile environment

Context:

Ensure actionable R&D in Data Science and AI:

-Research, test, and validate the contribution of innovative approaches (NLP, Computer Vision, Speech Recognition, Boosting) to data-driven business problems (fraud detection, semantic search, propensity scores, operational efficiency, model interpretation)

-Contribute, where applicable, to putting in place adequate solutions to address these issues using the internal platform for industrializing Python projects in production, integrating BizOps principles and extended exposure capabilities: Batch, Streaming, API, Webapp

Responsibilities

- Development of elegant and simple code for high-performance, reliable, and maintainable architectures and applications.

- Setup and maintenance of the CI/CD pipeline

- Setup of tests and coverage for the entire software architecture

- Production deployment and operation of machine learning pipelines in Python

- Monitoring of models in production with webapps (streamlit/dash)

- Training and production deployment of an identity theft detection model as part of fraud prevention:

-> Extraction and transformation of approximately 700 GB of data with PySpark

-> Inference in less than 500 ms with a boosting-type model

-> Streaming inference (1 million events per day)

- Production deployment of a check fraud detection model:

-> Conversion of SQL code to Python for near real-time inference (batch to micro-batch processing)

- Optimization: design of a Python solver (internal solution) to solve an optimization problem under a large number of non-linear constraints for marketing solicitations

- Extraction, manipulation, valuation & visualization of a large volume of data (> 700 GB)