Jacky Lam

Senior Data Engineer

EF Education First

Background

I’m Jacky, a Data Engineer based in New York. I got into data work somewhat by accident — but I’m glad I did.

My first real exposure was at the NYC Taxi & Limousine Commission, where I spent three years helping shape transportation policy for the city. Working with data that actually affected how 8 million people got around was what got me hooked on this field.

Before that, I studied Business Administration at Queens College and spent three years in the New York Army National Guard as an Air Battle Management System Operator. The military taught me how to think in systems and stay organized under pressure — turns out that’s pretty useful in data engineering too.

Between the Army and TLC, I consulted for the NYC Department of Parks and Recreation, managing data collection for Trees Count! 2015 — a citywide tree census. Honestly, at the time I was more focused on the people management side and didn’t think much about the dataset itself. Looking back, that was probably where the seed was planted.

I built this site to share things I’m working on and thinking about. If anything here catches your eye, feel free to reach out.

Experience

Data Engineer

Nuts.com

Jun 2021 – Present

ETL Pipeline Refactor & Cost Optimization: Refactored and optimized the core ETL data pipeline ingesting all website sales data—rewrote transformations and re-architected batch jobs to reduce infrastructure costs by 63% and cut job run times by 50%.
Product Cost Metrics: Drove a company-wide initiative to unlock visibility into product cost metrics by leading cross-functional collaboration between Finance, BI, and DevOps. Spearheaded project scoping, requirements gathering, and the end-to-end development of scalable data pipelines in Databricks and reporting infrastructure in Looker—enabling more accurate, actionable financial insights.
Databricks CI/CD Implementation & Job Reorganization: Architected and implemented CI/CD pipelines for Databricks by migrating notebooks and scripts into Git-based version control and reorganizing jobs by functional domain—standardizing deployments.
Data Modeling & dbt ELT Pipeline Architecture Architected and modeled raw sales data into efficient fact and dimension tables using dbt—optimizing schema design to reduce query times, unlock deeper visibility for business users, and accelerate data-driven insights.
Looker Style Guide Development & Standardization Partnered with BI and analytics stakeholders to develop and roll out a comprehensive Looker style guide—standardizing terminology, view structures, and design patterns to boost collaboration, reduce onboarding time, and ensure high-quality, consistent dashboards across the organization.
Stabilized BI Infrastructure with Blue/Green Deployment Engineered and deployed a Blue/Green data deployment strategy to stabilize BI infrastructure, reducing query latency errors by over 80% and significantly improving dashboard reliability and user trust.
Launched Foundational Analytics for New Digital Gifting Product Partnered with Product, Gifting, and Engineering teams to architect and implement the foundational reporting infrastructure for “Gift Drop”, a new product category—delivering end-to-end visibility into adoption, performance, and growth.
Established Scalable Metrics Framework for New Product Category Defined a unified, company-wide framework for the Gifting product category and collaborated with stakeholders to develop a comprehensive suite of performance metrics—establishing a consistent foundation for strategic analysis and long-term growth tracking.
Uncovered Key Customer Segments Through Journey Analytics Designed and executed customer journey analytics to identify key gifting segments, leveraging behavioral and transactional data. Enabled the commercial team to uncover loyalty gaps and develop data-driven strategies for targeted customer engagement and retention.

Senior Data Analyst

Motivate LLC.

Jul 2019 – Jun 2021 New York

Modernize Data Warehouse Architecture: Architected and implemented a scalable data warehouse solution post-Lyft acquisition, aligning legacy data systems with Lyft’s modern data infrastructure and ensuring seamless migration and integration.
Led Company-Wide BI Migration to Tableau: Led enterprise-wide BI tool migration from Domo to Tableau, designing and deploying optimized, consolidated dashboards that reduced redundancy and enhanced reporting accuracy. Established and managed a dedicated internal Slack channel to promote collaboration and rapid feedback with field leadership.
Enabled Near Real-Time Field Operations with GIS Dashboards:Developed geospatial heat map visualizations to support E-Bike Battery Swapping operations, providing operational leaders with real-time insights into field activity and improving logistical decision-making.
Managed a Cross-Functional Data Team: Managed and mentored a cross-functional data team of one Data Engineer and one Data Analyst. Facilitated daily stand-ups and sprint planning to ensure timely project delivery and alignment with organizational goals.

Senior Policy Analyst

NYC Taxi & Limousine Commission

Mar 2016 – Jul 2019 New York

Forecasted Ride Fare Dynamics Across Rideshare Providers and Taxis: Developed a multivariate linear regression model to forecast ride fares across Taxi, Uber, Lyft, Via, and Juno. Conducted comparative rate analysis to support a landmark for-hire driver income policy, influencing regulatory decisions.
Launched High-Impact GIS Tool Viewed Over 2.3M Times: Redesigned and digitized the Department of Transportation’s For-Hire Vehicle Relief Stand database using GIS tools, producing an interactive, driver-friendly map viewed over 2.3 million times since launch—dramatically improving public accessibility.
Defined Regulatory Data Standards and Vendor Compliance: Authored regulatory data specifications and led vendor compliance efforts. Partnered with IT to implement data ingestion pipelines, ensuring accurate and timely loading of for-hire vehicle datasets into production systems.

Street Tree Census Assistant Coordinator

NYC Department of Parks and Recreation

Mar 2015 – Mar 2016 New York

Team Leadership: Supervised a team of 25 staff members and trained volunteers in the data collection of over 660,000 street trees as part of NYC’s TreesCount! census initiative.
Asset Management: Served as the lead administrator for Histree, an asset management system by Nektar Inc., used to manage volunteer data, company assets, and asset logistics.
Stakeholder Engagement: Acted as the primary agency liaison for a public-private partnership with BMW, which donated 20 new BMW i3 vehicles to support data collection efforts during the first six months of the project.

Operations Manager

J. Trucking

Aug 2014 – Mar 2015 New York

Operations Oversight: Managed day-to-day operations for a small logistics firm with 7 employees; reported directly to the CEO and supported strategic business functions.
Client Account Management: Maintained and grew relationships with seven high-end retail clients, ensuring timely and reliable service delivery.
Process Improvement: Implemented new workflows and introduced cloud-based tools to enhance office efficiency and improve internal coordination.

Air Battle Management System Operator

New York Army National Guard

Mar 2013 – Mar 2016 New York/Oklahoma