Selective Prediction
An approach where a model refuses uncertain predictions and delegates to humans or other systems.
Selective prediction lets models refuse uncertain predictions – better to say "I don't know" than answer incorrectly.
Explanation
Selective prediction uses confidence thresholds or learned reject functions. The model answers only when sufficiently confident – the risk-coverage trade-off is explicitly controlled.
Marketing Relevance
For chatbots and support AI: Better to escalate to a human than answer incorrectly – reduces hallucination risk and increases trust.
Example
A medical diagnosis chatbot responds when uncertain: "I'm not confident enough – please consult a doctor."
Common Pitfalls
Too high threshold → too many abstentions (poor UX); too low → wrong answers slip through. Threshold must be tuned per segment.
Origin & History
Chow (1970) introduced the reject option for classifiers. El-Yaniv & Wiener (2010) formalized the risk-coverage trade-off. With LLMs, selective prediction became newly relevant from 2023 through abstention mechanisms.
Comparisons & Differences
Selective Prediction vs. Guardrails
Guardrails filter after generation; selective prediction decides before output whether to answer at all.
Selective Prediction vs. Human-in-the-Loop
HITL involves humans always; selective prediction escalates only on uncertainty – more efficient at high volume.