Skip to content

Fixed Windows UTF-8 console output for tree & table + tests#80

Open
Defiect wants to merge 2 commits into
jiacai2050:mainfrom
Defiect:windows-utf-8-fixes
Open

Fixed Windows UTF-8 console output for tree & table + tests#80
Defiect wants to merge 2 commits into
jiacai2050:mainfrom
Defiect:windows-utf-8-fixes

Conversation

@Defiect

@Defiect Defiect commented Apr 11, 2026

Copy link
Copy Markdown

Summary

This PR fixes broken console output (a.k.a. mojibake) on Windows for the UTF-8 box-drawing modes of tree, loc, and pretty-csv.

This adds a shared zigcli.term helper that temporarily switches an attached Windows console's output code page to UTF-8 for commands that emit non-ASCII border/tree glyphs by default, while leaving redirected output unchanged.

Non-Windows targets remain unchanged.

Reason for PR

When running the Windows executables under WSL interop / Windows Terminal, UTF-8 box-drawing characters were being reinterpreted through the console's active code page and showed up as mojibake.

Example:

  • Expected: ├──
  • Actual: Γö£ΓöÇΓöÇ

Redirected output was already correct, which pointed to console-layer translation rather than incorrect UTF-8 generation.

Changes

  • Added term.enableUtf8ConsoleOutput(file) and term.Utf8ConsoleOutput in src/term.zig
  • Restores the original Windows console output code page on exit when after our process changes it
  • Wired the helper into:
    • tree for .box and .dos modes
    • loc for non-ASCII separator modes
    • pretty-csv for non-ASCII separator modes
  • Added term docs for the new helper
  • Updated the Windows line-reading loop in loc to use takeDelimiter('\n') instead of takeDelimiterExclusive('\n')

Additional note on loc

While verifying this change, zig build test-loc was hanging on Windows in the LOC Zig/Python/Ruby test.

Note: this issue is unrelated to the stdout test issue (ziglang/zig#18111) and instead comes from the Zig 0.15.2 std.Io.Reader.takeDelimiterExclusive('\n') behavior on the Windows code path. Switching loc to takeDelimiter('\n') fixes the hang and matches the recommended usage for this Zig version.

Verification

Automated tests passed:

  • zig build test-tree
  • zig build test-pretty-csv
  • zig build test-loc
  • zig build test

Manual passes in WSL / Windows Terminal:

  • tree.exe renders UTF-8 tree glyphs correctly
  • loc.exe renders UTF-8 box borders correctly
  • pretty-csv.exe renders UTF-8 box borders correctly
  • ASCII modes still work as expected
  • redirected output still preserves valid UTF-8 bytes

Scope / behavior

  • Windows-only behavior change
  • No CLI flags changed
  • No change for redirected or piped output
  • No change for Linux/macOS behavior
  • pretty-table itself remains side-effect free; console setup stays in CLI entrypoints

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to temporarily enable UTF-8 output for Windows consoles using enableUtf8ConsoleOutput in src/term.zig. This functionality is integrated into the loc, pretty-csv, and tree binaries to support non-ASCII characters. Additionally, a fix was implemented in loc.zig to prevent a hang on Windows by replacing takeDelimiterExclusive with takeDelimiter. Feedback suggests adding a debug log or explanatory comment in Utf8ConsoleOutput.deinit when the restoration of the original console code page fails.

Comment thread src/term.zig

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses mojibake on Windows when emitting UTF-8 box-drawing characters by adding a shared terminal helper to temporarily switch an attached Windows console’s output code page to UTF-8, and wiring it into relevant CLI entrypoints.

Changes:

  • Add zigcli.term.enableUtf8ConsoleOutput(file) + term.Utf8ConsoleOutput guard (with unit tests) to best-effort switch/restore Windows console output code page.
  • Use the helper in tree, loc, and pretty-csv when non-ASCII border/tree modes are selected.
  • Update loc Windows line-reading loop to use takeDelimiter('\n') and document the new helper in docs/content/packages/term.org.

Reviewed changes

Copilot reviewed 2 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/term.zig Introduces UTF-8 console output guard/helper and tests for decision logic.
src/bin/tree.zig Enables UTF-8 console output for .box / .dos tree modes (+ small gating test).
src/bin/pretty-csv.zig Enables UTF-8 console output for non-ASCII table border styles.
src/bin/loc.zig Enables UTF-8 console output for non-ASCII separator modes; adjusts Windows line reading to avoid hang.
docs/content/packages/term.org Documents enableUtf8ConsoleOutput() usage and behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/term.zig
@jiacai2050

jiacai2050 commented Apr 11, 2026

Copy link
Copy Markdown
Owner

Please fix the AI reviews, and pass the CI.

@Defiect

Defiect commented Apr 14, 2026

Copy link
Copy Markdown
Author

Please fix the AI reviews, and pass the CI.

Done. CI and Release workflows all passed on my fork as well.

Comment thread src/term.zig
}
};

const Utf8ConsoleDecision = struct {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we combine Utf8ConsoleDecision and Utf8ConsoleOutput into one structure?

It make no sense to define two structs. A better API I could think of:

fn enableUtf8Console() ?ConsoleGuard

It will return null for non-windows OS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants