Skip to content

TabICL classification predict returns numeric labels without return_proba, strings with it #45

Closed

Description

@amrit110

What happens

The type of predictions in a classification predict depends on the backend and on return_proba:

Model return_proba: true return_proba: false
tabicl-v2 ["1.0"] (strings) [1.0] (numbers)
tabpfn-3.5 ["1.0"] ["1.0"]

Seen on the prod A40 deployment through the gateway on 2026-10-20, fitting train_y: [0, 1, 2] and predicting one row. A client that compares predictions with classes, or switches return_proba on and off, gets different types for the same answer. The gateway docs now warn about it (VectorInstitute/aieng-proxy-gateway#471).

Cause

TabICLBackend._predict (src/tabctx/backends/tabicl.py:530) returns model.predict(X_test).tolist() when return_proba is false, which keeps the label dtype (float for JSON numbers). With return_proba it derives labels from predict_proba as str(c) for c in model.classes_. The TabPFN backend (src/tabctx/backends/tabpfn.py:186) always takes the second route.

The TabICL route also goes against the backend protocol in backends/base.py, which says to derive predictions from probabilities rather than calling predict().

Fix

Make TabICL do what TabPFN does: always call predict_proba, take the argmax over [str(c) for c in model.classes_], and return probabilities only when asked. Regression stays numeric. Then add a test over both backends that classification predictions are strings drawn from classes, whatever return_proba is, and drop the exception from the gateway docs.

Labels also come back in float form ("1.0" for a JSON 1), because the JSON train_y is parsed as floats. That's a separate question: whether to keep integer labels as integers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High: next uparea/apiRequest and response contract, errors, uploadsbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions