Fixed Windows UTF-8 console output for tree & table + tests#80
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to temporarily enable UTF-8 output for Windows consoles using enableUtf8ConsoleOutput in src/term.zig. This functionality is integrated into the loc, pretty-csv, and tree binaries to support non-ASCII characters. Additionally, a fix was implemented in loc.zig to prevent a hang on Windows by replacing takeDelimiterExclusive with takeDelimiter. Feedback suggests adding a debug log or explanatory comment in Utf8ConsoleOutput.deinit when the restoration of the original console code page fails.
There was a problem hiding this comment.
Pull request overview
This PR addresses mojibake on Windows when emitting UTF-8 box-drawing characters by adding a shared terminal helper to temporarily switch an attached Windows console’s output code page to UTF-8, and wiring it into relevant CLI entrypoints.
Changes:
- Add
zigcli.term.enableUtf8ConsoleOutput(file)+term.Utf8ConsoleOutputguard (with unit tests) to best-effort switch/restore Windows console output code page. - Use the helper in
tree,loc, andpretty-csvwhen non-ASCII border/tree modes are selected. - Update
locWindows line-reading loop to usetakeDelimiter('\n')and document the new helper indocs/content/packages/term.org.
Reviewed changes
Copilot reviewed 2 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/term.zig |
Introduces UTF-8 console output guard/helper and tests for decision logic. |
src/bin/tree.zig |
Enables UTF-8 console output for .box / .dos tree modes (+ small gating test). |
src/bin/pretty-csv.zig |
Enables UTF-8 console output for non-ASCII table border styles. |
src/bin/loc.zig |
Enables UTF-8 console output for non-ASCII separator modes; adjusts Windows line reading to avoid hang. |
docs/content/packages/term.org |
Documents enableUtf8ConsoleOutput() usage and behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Please fix the AI reviews, and pass the CI. |
| } | ||
| }; | ||
|
|
||
| const Utf8ConsoleDecision = struct { |
There was a problem hiding this comment.
Could we combine Utf8ConsoleDecision and Utf8ConsoleOutput into one structure?
It make no sense to define two structs. A better API I could think of:
fn enableUtf8Console() ?ConsoleGuard
It will return null for non-windows OS.
Summary
This PR fixes broken console output (a.k.a. mojibake) on Windows for the UTF-8 box-drawing modes of
tree,loc, andpretty-csv.This adds a shared
zigcli.termhelper that temporarily switches an attached Windows console's output code page to UTF-8 for commands that emit non-ASCII border/tree glyphs by default, while leaving redirected output unchanged.Non-Windows targets remain unchanged.
Reason for PR
When running the Windows executables under WSL interop / Windows Terminal, UTF-8 box-drawing characters were being reinterpreted through the console's active code page and showed up as mojibake.
Example:
├──Γö£ΓöÇΓöÇRedirected output was already correct, which pointed to console-layer translation rather than incorrect UTF-8 generation.
Changes
term.enableUtf8ConsoleOutput(file)andterm.Utf8ConsoleOutputinsrc/term.zigtreefor.boxand.dosmodeslocfor non-ASCII separator modespretty-csvfor non-ASCII separator modestermdocs for the new helperlocto usetakeDelimiter('\n')instead oftakeDelimiterExclusive('\n')Additional note on
locWhile verifying this change,
zig build test-locwas hanging on Windows in theLOC Zig/Python/Rubytest.Note: this issue is unrelated to the stdout test issue (ziglang/zig#18111) and instead comes from the Zig 0.15.2
std.Io.Reader.takeDelimiterExclusive('\n')behavior on the Windows code path. SwitchingloctotakeDelimiter('\n')fixes the hang and matches the recommended usage for this Zig version.Verification
Automated tests passed:
zig build test-treezig build test-pretty-csvzig build test-loczig build testManual passes in WSL / Windows Terminal:
tree.exerenders UTF-8 tree glyphs correctlyloc.exerenders UTF-8 box borders correctlypretty-csv.exerenders UTF-8 box borders correctlyScope / behavior
pretty-tableitself remains side-effect free; console setup stays in CLI entrypoints