Description

I help companies turn raw, messy, or siloed data into reliable, production-grade pipelines and analytical products. With 15 years of experience — the last 5 focused on data engineering and data science — I work across the full data stack: from ingestion and quality to modelling, streaming, and machine learning.

My core stack is Python, PySpark, Databricks, dbt and Delta Lake, with hands-on experience in batch and near-real-time pipelines, time series forecasting and Kubernetes-based data platform design.

I've delivered solutions for clients in banking, energy, healthcare, and oil & gas — both as a hands-on engineer and as a solutions architect. I'm comfortable owning a project end-to-end: scoping, building, and documenting — or plugging into an existing team as a senior technical resource.

If you need someone who can bridge the gap between business requirements and data infrastructure, I'm your person.

Industry field of expertise

Languages

Portuguese
Native or bilingual
English
Fluent
Spanish
Conversational

Workplace preferences

Remote only

Primarily works remotely

Banco Montepio
Data Engineer
February 2024 - Today (2 years and 6 months)
Portugal
• • Refactored legacy SAS models into PySpark and Delta Lake pipelines, applying survivorship rules to resolve entity conflicts across banking master data.
• • Performed gap analysis between source system outputs and regulatory reporting layers, flagging and correcting discrepancies ahead of submission deadlines.
• • Built contract-based validation with Pydantic — enforcing field completeness and type conformity at ingestion, catching quality issues at source rather than downstream.
• • Maintained audit-ready datasets in line with Bank of Portugal and ECB directives, documenting each quality rule as a reproducible SOP per regulatory workstream.
• • Improved pipeline throughput through Spark tuning — partition strategies, broadcast joins, and removal of full table scans on large Delta tables.
• • Absorbed undocumented Shadow IT sources (Excel extracts, local databases), reverse-engineering their structure and formalising them into governed, migration-ready assets.
DBT pyspark Kubernetes Cloud data lakehouse
Joyn Energy Management
Data Engineer – IoT & Streaming
May 2025 - February 2026 (9 months)
Portugal
• • Designed and implemented an Azure-based near-real-time ingestion pipeline for Victron photovoltaic devices, processing JSON telemetry via Azure Event Hubs and Azure Functions into a Delta Live Tables bronze layer.
• • Built structured streaming pipelines with Auto Loader, enabling continuous ingestion of IoT sensor data from commercial buildings equipped with solar panels and EV chargers.
• • Developed ML-based approaches for solar energy forecasting and EV charging optimisation, applying time series modelling to energy production and consumption patterns.
• • Monitored pipeline reliability and data freshness across streaming layers, implementing data quality checks to detect anomalies in device telemetry before downstream processing.
EDP — CGI
Data Engineer
July 2022 - February 2024 (1 year and 7 months)
Portugal
• • Guaranteed billing accuracy for individual and corporate energy clients by tracing metering records end-to-end — from smart meter ingestion to invoice-ready output.
• • Harmonised large volumes of historical measurement data during SAS-to-Databricks migration, resolving format inconsistencies, unit mismatches, and duplicate entries.
• • Tuned high-throughput Spark jobs for billing data processing, cutting execution latency and reducing compute costs on high-frequency energy datasets.
• • Wrote targeted SQL audit queries to trace anomalies across interconnected metering and billing systems, directly supporting financial accuracy reviews.
• • Translated domain requirements into a reusable set of data quality checks — documented and applied consistently across all energy measurement pipelines.