// lead data engineer

I build the pipelines and ontologies that make data usable by people and AI.

I'm the lead data engineer on the U.S. Marine Corps Logistics Analytics Data Services team. Day to day I write Python, PySpark and SQL in Databricks and model the results as a Palantir Foundry ontology that analysts can query with AI.

fetch('bobby-status.json')    .then((response) => response.json())    .then((data) => {        console.log(data.status);    });// For those who can't view the console// it says "building ontologies"

Featured work

Ground equipment ontology for the U.S. Marine Corps

I designed and built this as its primary developer. It turns 15+ years of flat, denormalized outbound data feeds from the enterprise's ERP (enterprise resource planning) systems into a connected, time-aware Foundry ontology that analysts can explore through Foundry AIP.

15+ yrsof history, back to ~2010
20+object types in the ontology
50+links between objects
Source Databricks Foundry Ontology Analysts Daily ERP data feeds Historical tables (~2010 →) ① Derive primary keys ② Order records in time ③ Split objects / events ④ SCD-2 effective dates 20+ object types 50+ links Object-event model Plain-language descriptions via AIP & AI-FDE
Simplified architecture view.
  1. Rebuilding missing keys. The daily feed arrives with most source-system identifiers, including primary keys, stripped out. I wrote PySpark and SQL that derives a stable unique key for every record, which was the hardest part of the project.
  2. Restoring history. Records are ordered in time within their windows, so each piece of equipment has a reliable timeline instead of a pile of daily snapshots.
  3. Modeling objects and events. The flat tables are split into objects and the events that happen to them. History tables (SCD Type 2) carry derived effective_from / effective_to dates, so point-in-time questions become simple filters.
  4. Making it AI-ready. Every object type and property has a plain-language description. That semantic layer lets analysts and Foundry's AI tools understand what the data means, so people can work with it by asking questions instead of decoding tables.
  5. Building with AI. I used the DoD's GenAI.mil LLM platform throughout to plan the data model and to draft, refine and review transformations.
PythonPySparkSQLDatabricksDelta LakePalantir FoundryFoundry AIPOntology designSCD Type 2GenAI.mil

Personal projects

Expectation-driven data engineering

A public pipeline over New York's congestion pricing data. It records what a metric is expected to do before the data arrives, then scores each expectation automatically and shows where any gap comes from. I built it to show how I work, since most of my day job can't be published.

1.07Btaxi and ride-hail trips processed
13expectations recorded, frozen and scored
3 of 3published MTA figures reproduced within 0.2 points
PySparkApache IcebergAirflowDockerData quality checksGitHub Actions CIpydanticLLM elicitationClaude Code

A toy agent for the same warehouse

An experiment on which written context helps a model turn a plain-language question into the right SQL: table descriptions, table selection, or short narratives of how each process moves through the tables. Two small local models, with a prediction for every answer committed before the run.

144answers scored against reference results
8 to 10of 18 correct on the larger model once process narratives were added
118 of 144predictions matched the outcome; predicting failure everywhere would score 109
Text-to-SQLConversational reportingLLM evaluationOllamaSpark SQLData catalog descriptionsPythonClaude Code

More work

Migrating a team's whole data stack

Moved my team off a legacy AWS GovCloud stack (NiFi ETL, Redshift, Athena, QuickSight) onto Databricks and Foundry. I designed the target architecture, pipeline patterns and CI/CD, with no loss of capability along the way.

AWS GovCloudDatabricksCI/CDMedallion architecture

Training data for large language models: Scale AI

Wrote and evaluated Python and SQL solutions used to train and test LLMs. I was promoted to lead reviewer within three months and reviewed other contributors' code for correctness and quality.

LLM training dataCode reviewPythonSQL

How I work with AI

Outside work, I spent two months building several Python projects with Claude as a collaborator, including a numerical library and a command-line text pipeline built on word embeddings. What carried over to my day job is the method: I set the direction, and I keep a running log of what's established, what's been tried and what failed, so each session starts with full context. I check results against tests and baselines before I trust them, and the core library runs its own verification checks.

LLM-assisted developmentPythonNumPyEmbeddingsTesting

Experience

2024 – now
Lead Data Engineer, Marine Corps Systems CommandLogistics Analytics Data Services (LADS). Federal civilian, Computer Scientist (1550).
2023 – 2024
AI Training Contractor, promoted to Lead Reviewer, Scale AI (Remotasks)Code and SQL training data for LLMs.
2020 – 2023
Independent Author & PublisherWriting and self-publishing; building my own sites brought me back to software.
2019 – 2020
Software Engineer, Epic SystemsFull-stack C# and TypeScript/React for the Health Information Management suite.
2015 – 2019
B.S. Computer Science, UNC WilmingtonSystems concentration, cum laude.
2010 – 2015
U.S. Marine CorpsCorporal, rifle team leader and Combat Marksmanship Coach, 2nd Battalion, 9th Marines.

Skills

Languages & practices

Python (PySpark, pandas), SQL, C#, TypeScript. Automated testing, code review, CI/CD, Git

Data platforms

Databricks, Delta Lake, Palantir Foundry, Apache Iceberg, Apache Airflow, AWS GovCloud (Redshift, Athena, S3), Apache NiFi

Modeling

Ontology and knowledge graph design, entity and relationship modeling, object-event temporal models, SCD Type 2, star schemas

AI & LLMs

Foundry AIP and AI-FDE, semantic layers for natural-language querying, LLM-assisted development (GenAI.mil, Claude), word embeddings, LLM training data