Define prediction time before selecting features
A field is useful only if it exists when the decision is made.
Put a clock beside the task
Suppose a fictional service team wants to flag support tickets likely to miss a response target. Decide whether the prediction happens when the ticket arrives or after the first reply. That choice changes which fields are legitimate. A final resolution code is unavailable at arrival, even if it appears neatly in the exported training table.
Create an availability ledger
For every proposed feature, record its source, earliest availability, update delay and meaning. Distinguish event time from the time a system recorded an event. Ask whether a field is corrected later. If historical exports contain the corrected value, reconstructing an earlier decision can require a separate snapshot rather than a filter on the latest table.
Check the split against deployment
A random split can be inappropriate when the intended use is predicting a later period or serving new organisations. Choose a split that reflects that use. Keep records from the same entity together where dependence would otherwise make evaluation misleading. Document the trade-off: a stricter split can leave fewer examples, but it answers a more relevant question.
Retain one worked record
Write out the feature values for a single fictional or safely de-identified prediction at the chosen time. Ask an operator whether those values were actually available then. This low-tech review often catches a timing assumption before the team invests in a complicated model. It supplements, rather than replaces, a systematic leakage audit.
References provide background. The working methods and fictional examples here are our editorial guidance, not an endorsement by the referenced organisations.
Put this into practice: open the worksheet →