Delivery to Bowl: The Whole Pipeline

The lesson

Half past 4, one bulb burning

It is 4:30 in the morning, and most of the street is shut. Mine is the only kitchen that has its lights on, and I am writing the day’s delivery list. Behind me, the fire is not lit. There is no flame at this hour. The famous part of my day is the searing of the lamb over a live flame. That sear is what everyone remembers, and it is actually the fastest part of making a bowl of Irish stew. Everything before it is tireless work you never see, under delivery-lorry lights and over a sink, and it takes up most of the day. Follow my whole day, and you would have walked through what every AI project goes through. There are 7 steps, and one bowl.

Decide the dish before you shop

Step one happened years ago, when I chose the problem to solve. I wanted to serve Irish stew for a living. That choice rules everything after it. An AI project starts the same way, by defining its problem. Step two is the morning delivery. That is collecting the data. Step three is the sink. Every carrot gets washed and trimmed, every potato scrubbed, every bad bit cut away. Cleaning data is the same joyless job, because the world delivers gritty vegetables and messy records. Step four is the stock in my big pot. Hours of quiet simmer turn lamb bones and trimmings into a rich stock. This is what feature engineering is. You boil raw data down to the good stuff a model can taste. Step five is picking the stew recipe. That is choosing the algorithm. Step six is the sear over the flame. That is training, where the machine learns from examples and adjusts itself. Step seven is taste and adjust. That is evaluation, spoon by spoon.

Then I serve the bowl of stew to the customer, and I keep listening. A model that goes live is watched the same way. Customers drift. A model that was right in January can be wrong by June. The listening helps me tweak my recipe, and we go straight back to step one. That is why engineers draw the whole thing as a circle.

The same 7 steps at a bank

Run the same 7 steps at a bank. The only change is the uniform. Decide the problem: which card payments are fraud. Collect: years of transactions. Clean: repair the missing and mislabelled records. Features: boil them down to what matters, such as the hour and the amount. Algorithm: choose the sorter. Train: let it learn. Evaluate: test it on payments it has never seen, and see if it managed to predict the fraud.

Say the 7 steps, no notes

Now you walk the 7 steps yourself. Say the 7 steps to yourself. I give you three seconds.

Pause here for three seconds.

6 out of 7 beats most project meetings. Now the question that matters. Across the whole day, where did the hours go? Three seconds.

Pause here for three seconds.

Most of the day went into the hours before the sear. Preparation is the real work, and the flame is the smallest part of it. Every AI project runs the same way. The data work consumes the most time. The glamorous sear takes a few minutes. Most projects are not lost at training. They are lost at the sink.