Skip to content

Fix Unicode width, precision and escaping for wide strings - #4970

Closed
tianrking wants to merge 2 commits into
fmtlib:mainfrom
tianrking:codex/unicode-wide-string-formatting
Closed

tianrking wants to merge 2 commits into
fmtlib:mainfrom
tianrking:codex/unicode-wide-string-formatting

Conversation

@tianrking

@tianrking tianrking commented Oct 4, 2026 •

Copy link
Copy Markdown

Fix Unicode display width, precision and debug escaping for wide strings. UTF-16 surrogate pairs are decoded together; unpaired surrogates keep their individual escapes.

The wide-character codepoint traversal and string-formatting implementation now live in xchar.h. Narrow-string formatting remains in format.h, with the existing escape scanner available to headers that do not include xchar.h.

Tests focus on Unicode width and precision, surrogate pairs, and debug escaping. The extra char8_t, range, output-iterator and dynamic-width cases have been removed.

Validation with MSVC 19.44, x64, C++20 Debug: full CMake build passes and all 45 xchar tests pass. CTest passes 20 of 22 test programs; chrono_test.locale and unicode_test.legacy_locale fail with the Windows locale errors already recorded in the original validation. Changed code passes clang-format 20 and git diff --check.

@tianrking
tianrking requested a review from vitaut as a code owner October 4, 2026 18:16

@vitaut vitaut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Could the wide-character-specific code be moved to xchar.h? In particular, the new for_each_codepoint overload and wide-string formatting logic.

@vitaut

vitaut commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Also the tests seem excessive for this change. Could you trim them down to focus on the actual fixes: Unicode width and precision, surrogate pairs, and debug escaping? In particular, the char8_t, range formatting, output iterator, and dynamic width tests don't seem necessary.

@vitaut

vitaut commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Do you plan to update this PR or shall we close it for now?

@vitaut vitaut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there are CI failures.

@vitaut

vitaut commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Closing for now but feel free to reopen once CI issues are fixed.

@vitaut vitaut closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants