What I've built.

// Problem

A constantly growing ~900 TB data lake, with multiple squads producing ETLs, had purely reactive failure detection — issues only surfaced when a business user reported incorrect or missing data, often days after the original failure.

// Solution

• → Led a squad of 5 to 8 engineers on lake architecture and governance, standardizing medallion layers and Glue Workflows/Jobs.
• → Built observability from scratch: a shared Python library for Glue jobs, with Lambda-triggered alerts in Microsoft Teams (error details + direct CloudWatch link).
• → Standardized triage: every failure reaches the team with enough context for immediate diagnosis.

// Problem

Customer, policy, and complaint data had to reach customer service systems (CRM) in near real time, while also meeting regulatory data-exchange requirements between insurers (Open Insurance/SUSEP).

// Solution

• → Maintained and evolved a Change Data Capture pipeline with Apache Kafka, capturing changes to customer, policy, and complaint data at the source.
• → Served the processed data through DynamoDB, exposed via APIs consumed by the customer service CRM.
• → Managed the Open Insurance integration: receiving data from other insurers and submitting data in the regulatory standard required by SUSEP.

// Problem

The company had never put an AI project into production. Complaint analysis relied on manually querying data scattered across multiple systems (customer, policy, complaint history), with no systematic decision support for analysts.

// Solution

• → Led the data engineering and MLOps work: aggregating customer, policy, and complaint data into a unified view.
• → Integrated the pipeline with the classification model developed by the data science team.
• → Delivered grounded recommendations in a Power BI dashboard, with the supporting data behind each suggestion.

// Problem

A data science study had validated promising hyperparameters for document analysis with open-source LLMs, but there was no production path: no prediction API, no model/version orchestration, no infrastructure to serve external customers.

// Solution

• → Built the production pipeline for the 3 models (Gemma, Llama, and Qwen), including orchestration of which model/version serves each prediction.
• → Developed prediction serving and management APIs, decoupling the data science team from infrastructure complexity.
• → Manage the cloud infrastructure serving the models to external customers, and integrated generative AI (Claude and agents) into the engineering workflow — code review, quality, security, and testing.

// Problem

A large telecom carrier (~80 million customers) needed to consolidate customer, campaign, and segment data from heterogeneous batch and streaming sources, and reliably activate multichannel communication journeys.

// Solution

• → Modeled XDM schemas with schema evolution and built batch (BigQuery/CSV) and streaming ingestion, configuring client-side Kafka integrated with AEP.
• → Created multichannel activation journeys in Adobe Journey Optimizer (email, push, and SMS), including domain IP warming strategies.
• → Built complementary data pipelines with Airflow and Redshift across AWS and GCP environments.

// Problem

Similarity and segmentation variable calculations ran on BigQuery with executions that took days, prone to frequent failures and restarts, driving up cloud costs and delaying variable availability for models.

// Solution

• → Refactored the variable calculations from BigQuery to Spark/Scala, running on a 10 to 15 node Dataproc cluster.
• → Created an internal Feature Store for registering and managing models and variables, with OAuth 2.0 authentication integrated with GCP.
• → Developed REST APIs in Flask and Airflow DAGs to orchestrate processing, with CI/CD deployments.