How I work, on one project
52 million rows of machine logs
Tool-change logs from CNC lathes at two plants. Before training any model, I wanted to establish what the "remaining life" counter actually counted.
Keep scrolling
Check 2: state what the data cannot show
The counter measured output, not wear.
−26%per-tool error on remaining life
About 15% of the gain came from switching to XGBoost. The rest came from excluding one machine whose counter had frozen. A naive baseline scores 0.257.
What the counter trackedSchematic
Counter: pieces made against a quotaTool wear: not recorded in the data
Check 1: validate it the way it will be used
Keep each tool cycle on one side of the split
Readings within one cycle are near-duplicates of each other, so a random split leaks the answer into the test set. I grouped the split by cycle and added a time-based split. The time-based split rejected a model that twenty random splits had accepted.
Check 3: make it verifiable
Then build it with tests
25test suites, support assistant
163tests, drawing reader
27/27multiplayer QA checks, game
That was one project.
The full case covers the method, the validation and what I recommended at handover.