Raphaël Benz

🚀 Transversal consulting and GenAI audit role within BPCE, supporting over thirty internal projects (Risk, Finance, Legal, IT, Operations). Evaluation of LLM architectures, pipeline robustness, data compliance, technological choices, and industrialization best practices. Strategic contribution to the structuring of the group's GenAI governance.

🛠 Participation in the development of NOVA, the internal low-code/no-code platform dedicated to creating complex GenAI workflows: multi-step RAG, orchestrated agents, specialized tools, hybrid pipelines with business rules. Co-construction of reusable modules, usage guidelines, and integration of security components (audit, traceability, sensitive prompt management).

🎯 Lead of a team of 5 Data Scientists and 2 Software Engineers in the design of an internal LLM Judge, an automatic LLM response evaluation tool usable by both DS and business teams. Definition and implementation of two core metrics (correctness and groundedness) as well as a modular system allowing easy addition of new evaluation criteria specific to use cases.

📊 Development of a retriever evaluation module, including analysis of passage relevance, contextual recall, robustness to query variations, and measurement of consistency between retrieval and generation. Integration into existing workflows to ensure a complete diagnosis of RAG systems.

🧩 Construction of the back-end and user interface for the evaluation tool (API, orchestration, database, analysis-oriented UI), enabling full autonomy for business teams and rapid adoption. Already used in over 10 GenAI projects within the group.

🔧 Techs: Python, FastAPI, Docker, GitLab CI, Vector DB (FAISS/Qdrant), HuggingFace, LangChain/LlamaIndex, React, ElasticSearch, Kubernetes, SQL, internal monitoring, LLMs on-premise / private cloud.

🚀 Development of SpreadAuto, a tool for automatic extraction of datapoints from complex financial documents (balance sheet, income statement, cash-flow, detailed annexes). Design of a complete non-LLM pipeline, relying on YOLO for zone detection, Camelot for tabular reconstruction, OCR, business rules, and dictionary matching. The tool is now in production and used daily by 30+ analysts and Risk/Finance teams.

🧪 Implementation of TDD, clean architecture principles, and a GitLab CI/CD workflow strengthening code and model reliability. Construction of a documentary testing system with real cases, automatic performance tracking, structured logging, and operational monitoring to prevent regressions.

📦 Industrialization of the ML chain: packaging via Docker, deployments on internal infrastructure, management of GPU/CPU resources, orchestration of extraction services, and exposure of secure APIs consumed by other teams. Optimization of inference times and automation of release cycles.

🔍 Design of a RAG system for semantic datapoint extraction, independent of document type or format. Integration of internal embeddings, vector indexing (FAISS/Qdrant), business rules, and fallback strategies. Development of specialized modules for term sheets, valuation reports, and energy diagnoses, offering extractions adapted to different business contexts.

🔐 Deployment and optimization of local LLMs to comply with GDPR and banking constraints: quantization, memory configuration, creation of an internal inference service exposed via API. Implementation of an architecture ensuring sensitive data never leaves the BNP perimeter.

🔧 Techs: Python, FastAPI, Docker, GitLab CI, YOLO, Camelot, PyMuPDF, Tesseract, FAISS/Qdrant, Pytorch, ElasticSearch, Airflow/Kubeflow, SQL, Linux, on-prem infra.

🚀 Development of an ML model for La Plateforme Du Bâtiment, to forecast customer visit evolution using BG-NBD and other transformed customer data. Currently in production.

📊 Calculation of departmental market shares using linear models, relying on internal and scraped data from INSEE. Indirect performance measures (unsupervised) show an error margin of 10%.

🧩 Segmentation of the customer base for DSC -LightGBM Classifier- and construction of feedback loops for labeling for continuous improvement and performance monitoring.

🎯 Prediction of the commercial potential of the customer base + creation of a customer consumption embedding via Variational AutoEncoder -used subsequently in other ML models-.

🔧 Development of a testing and performance monitoring architecture for ML projects in production.

🔬 Techs: Python, Azure, CDSW, Impala, SQL, HDFS, Spark, Tensorflow, Scikit Learn, Pytorch.