Skip to content

Accept missing values (null / empty CSV cells) in fit and predict #61

Closed

Description

@amrit110

Real tables have missing values, and both TabPFN and TabICL handle NaN natively. Today there's no way to send one:

  • JSON: train_X/test_X are list[list[float]], so null gets a 422. A NaN literal isn't valid JSON, and the gateway's orjson parser answers 400.
  • CSV upload: np.loadtxt rejects empty cells. The text nan does parse.

Users must impute before sending, which loses information the models would use. The real-data eval (#59) had to median-fill Titanic, about 3% of cells.

Proposal:

  • Accept float | None in train_X/test_X and map None to NaN.
  • In CSV, treat empty cells, NA and nan as NaN.
  • The Parquet path (see the large-uploads issue) carries nulls natively.
  • Add tests for both backends: NaN in train only, in test only, and a whole column missing.

Activity

  1. added a commit that references this issue on Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions