LLM Classification Is Feature Engineering
Argues LLM classifiers should feed a downstream logistic regression, yielding calibration, threshold control, and principled use of structured covariates.
The post contends that LLM-as-classifier setups suffer from poorly calibrated hard labels, opaque use of prompt context and structured data, and weak interpretability. Wrapping the LLM verdict as a feature in a logistic regression restores calibrated probabilities, precision–recall threshold control, and the ability to incorporate additional covariates. Further gains can come from more training data, richer features such as log probabilities and subverdicts, and swapping in downstream models like xgboost or neural networks. An irony detection test case illustrates the approach.
32