Simon Roburin

Ongoing Projects

• Confidentiality & Frugality of LLMs
• Scientific Consulting for BeinK for Image Generation

Confidentiality & Frugality of LLMs

• Design of a method to amplify differential privacy (DP) using missing data mechanisms (MCAR/MAR):-18%training costs with lower ε-DP and marginal performance reduction:
• Experiments onLLM 7B/13B;
• Article in progress and code to be open-sourced.

Scientific Consulting for BeinK for Image Generation

• Fine-tuning strategies: LoRA/QLoRA, choice of layers to adapt, rank size, data selection, prompt conditioning, overfitting control.
• Design of a ready-to-use evaluation framework (test datasets, prompts, human evaluation metrics) and decision grid (quality, GPU costs, latency, safety).
• Synthetic tech radar (recent papers/tools) → benefits, limitations, risks.

Projects

• Visual Reasoning for VLMs in Real-World Conditions
• Real-time Ice Hockey Match Tracking

Visual Reasoning for VLMs

• Creation of a reasoning benchmark for VLMs:DrivingVQA(https://huggingface.co/datasets/EPFL-DrivingVQA/DrivingVQA):+3.1 ptsvs baseline onDrivingVQA&+6 ptsvs baseline onAOKVQA;
• Design of a new visual reasoning method for VLMs;
• Open-source release of the code https://github.com/vita-epfl/RIV-CoT and reproducible evaluation protocol

Real-time Ice Hockey Match Tracking

• Dataset creation: 32 professional match videos, multi-camera with bounding box annotations on players;
• Tracking model design: YOLOv5 detection + OC-SORT association + AFLink to reduce re-identification errors (ID switches):+14 pts HOTAvs Dartfish baseline & -50% ID switchesvs Dartfish baseline;
• Annotation cost reduction: 1/10 sampling study maintaining comparable performance (documented prototype & evaluation protocol).

Projects Led

• Human Resources— Design of an LLM pipeline for CV ↔ mission matching with RAG (LangChain): skill extraction & normalization, semantic search, explainable ranking, decision traceability. Privacy-preserving system (DP-SGD / federated learning), data stream encryption, access management.
• Electricity Provider— Extraction & classification of material test reports into a structured database: robust OCR (tables/figures), parsing, data schema, quality controls, KPIs (precision, recall), and integration API.
• Network Operator— Confidential training pipeline on sensitive data: DP-SGD, federated learning, encryption in transit/at rest, logging, utility ↔ confidentiality evaluation protocol (ε, δ) for compliance.