Cyrillic Paths in PowerShell and ssh: the Encoding Trap

01What you'll learn

• Why a Cyrillic path "looks fine" on screen while the file check says "does not exist".
• What differs between Windows PowerShell 5.1 and PowerShell 7 when it comes to encoding.
• How to check in 30 seconds what is set on your machine and how to fix output capture.
• How to write and read Cyrillic files without loss, and why a script needs a BOM in 5.1.
• How to carry paths as base64 and how to make ssh and Python speak UTF-8.
• One safety rule before every delete.

02Before you start

• A Windows computer with PowerShell — at least the built-in Windows PowerShell 5.1 (PowerShell 7 is optional).
• The OpenSSH client (ssh) and access to a Linux server — for example a machine of the NVIDIA GB10 class with DGX OS.
• Python 3 on the server (for the base64 examples) — on Ubuntu it is there by default.
• Nothing is deleted in this lesson. All examples use an invented path — do not run a delete on your own files until you have passed "Check".

03Steps

1. See the trap with your own eyes

   The script below is pure ASCII (the word is built from character codes) so that it depends on nothing. cmd /c type is an external program that returns UTF-8 bytes — exactly like ssh when the server sends a name with Cyrillic.

   powershell · on your computercopy

   # the word "Документи" ("Documents") built from character codes only
   $c = [string]::new([char[]](0x414,0x43E,0x43A,0x443,0x43C,0x435,0x43D,0x442,0x438))
   $t = Join-Path $env:TEMP 'enc_test.txt'
   # UTF-8 bytes, the way Linux sends them
   [IO.File]::WriteAllBytes($t, [Text.Encoding]::UTF8.GetBytes($c))
   $x = cmd /c type $t          # external program, its output is captured
   "before: $($c.Length) chars · captured: $($x.Length) chars · equal: $($x -eq $c)"
   Remove-Item $t

   On our machine (Windows 11, Windows PowerShell 5.1, code page 437) the result was 9 → 18 characters, "equal: False". Every letter doubled, because its two bytes were read as two characters. The console may still be able to "draw" them — the internal string, though, is already different.

2. Check what is set on your machine

   powershellcopy

   $PSVersionTable.PSVersion
   $OutputEncoding.WebName
   [Console]::OutputEncoding.WebName
   [Console]::InputEncoding.WebName
   chcp

   On ours: 5.1.26100.…, us-ascii, IBM437, IBM437 and "Active code page: 437". Yours may differ — it depends on the language and settings of Windows. What each line means:

   Setting	What it controls	Default (per the documentation)
   $OutputEncoding	How PowerShell sends text to an external program (pipe, ssh)	Windows PowerShell 5.1: ASCII · PowerShell 7: UTF-8
   [Console]::OutputEncoding	How the console translates the output of external programs	The console code page, from the system locale (.NET). ⚠️ We have not checked the PowerShell 7 behaviour — run the command and look
   chcp	The active console code page	Depends on the system (437, 850, 866…); 65001 is UTF-8

   ASCII in $OutputEncoding means every Cyrillic letter sent to an external program from 5.1 turns into ? — irreversibly. Capturing from the program is the other end of the same trap.

3. Fix the output capture

   Tell the console that external programs speak UTF-8. This is for the current session (it does not change Windows):

   powershellcopy

   [Console]::InputEncoding  = [Text.UTF8Encoding]::new($false)
   [Console]::OutputEncoding = [Text.UTF8Encoding]::new($false)
   $OutputEncoding = [Console]::OutputEncoding

   Repeat the test from step 1: on our machine, after this setting the captured text became equal to the original ("equal: True"). The PowerShell documentation advises that $OutputEncoding should match [Console]::InputEncoding, so we set them together. If you want this for every session, put the lines in your profile — but then it applies to everything you launch from there.

   ⚠️

   What about chcp 65001?

   It is often recommended as the matching switch on the console itself. Microsoft's chcp page does not include it in its table of code pages, and we have not tested it — so we offer it as an idea to try, not as a fix.

4. Write and read files with an explicit encoding

   This is where the two versions differ most. We measured on 5.1; for 7 we follow the documentation.

   Command	Windows PowerShell 5.1	PowerShell 7
   Set-Content without -Encoding	The system's ANSI code page — on ours Cyrillic became 3F (?)	UTF-8 without BOM
   Out-File and >	UTF-16 with BOM (measured: first bytes FF FE)	UTF-8 without BOM
   -Encoding ascii	every non-ASCII character → ?	the same
   -Encoding utf8	UTF-8 with BOM (measured: EF BB BF)	UTF-8 without BOM; utf8BOM is a separate value

   powershellcopy

   # write and read with an explicit encoding — works in both versions
   Set-Content -Path .\notes.txt -Value $text -Encoding utf8
   Get-Content -Path .\notes.txt -Encoding utf8
   
   # optional: UTF-8 by default for every cmdlet in the session
   $PSDefaultParameterValues['*:Encoding'] = 'utf8'

   Microsoft's documentation also reminds us about the BOM: for best compatibility UTF-8 files should be without a BOM, because Unix tools (cat, sed, awk) do not understand it. So: a file that Linux programs will read, write in PowerShell 7 with utf8, or in 5.1 with [IO.File]::WriteAllText($path, $text, [Text.UTF8Encoding]::new($false)).

5. A script with Cyrillic: UTF-8 with BOM for 5.1

   Windows PowerShell 5.1 reads a script without a BOM as ANSI. So a script with Cyrillic written in an editor that keeps UTF-8 without BOM (as VS Code does) breaks its strings or fails to parse (Unexpected token, string is missing the terminator). According to Microsoft, if the script has non-ASCII characters, save it as UTF-8 with BOM. Do not pass long scripts as a single string through -Command — save them to a file and use -File, or encode them in base64 (step 6).

6. Carry the path as base64

   Why: base64 is pure ASCII and goes through any pipe, code page and file unchanged. You encode on the UTF-8 side, carry ASCII, and decode to a real Unicode string right before use. The path in the example is invented.

   bash · on the servercopy

   python3 - <<'EOF'
   import base64
   p = "Документи/Отчети/годишен-отчет.txt"
   print(base64.b64encode(p.encode("utf-8")).decode("ascii"))
   EOF
   # 0JTQvtC60YPQvNC10L3RgtC4L9Ce0YLRh9C10YLQuC/Qs9C+0LTQuNGI0LXQvS3QvtGC0YfQtdGCLnR4dA==

   powershell · on your computercopy

   function ConvertFrom-B64([string]$b) {
     [Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($b))
   }
   $path = ConvertFrom-B64 '0JTQvtC60YPQvNC10L3RgtC4L9Ce0YLRh9C10YLQuC/Qs9C+0LTQuNGI0LXQvS3QvtGC0YfQtdGCLnR4dA=='
   $path                      # a correct Unicode string, in memory
   # for a local file: always -LiteralPath
   # Get-Item -LiteralPath $path

   In the other direction — a script for the server with Cyrillic inside — you encode the script to base64 on your computer and pass it so that only ASCII crosses the pipe:

   powershell · on your computercopy

   $b64 = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($script))
   ssh <user>@<server-address> "echo $b64 | base64 -d | bash"

   We checked the round trip (Linux → base64 → decoding in PowerShell 5.1) on an invented path: the result is the same string. The ssh connection itself was not run (see "What we have not run ourselves").

7. Set up the locale and Python on the server

   Two things to remember. First, OpenSSH sends no environment variables by default (SendEnv) and the server accepts none (AcceptEnv) — your locale does not travel by itself. ⚠️ Distributions often change this in their configuration files; it is not in the official OpenSSH documentation, so check your own machine. The simplest approach is to set the locale in the command itself.

   bash · on the servercopy

   locale -a | grep -i utf         # is C.UTF-8 there?
   LC_ALL=C.UTF-8 python3 -c "import sys; print(sys.stdout.encoding)"

   Second, Python has its own UTF-8 mode. According to the documentation it switches on automatically when the locale is C or POSIX. We checked in the sandbox:

   bashcopy

   LC_ALL=C python3 -c "import sys; print(sys.stdout.encoding, sys.flags.utf8_mode)"
   # utf-8 1
   LC_ALL=C PYTHONUTF8=0 python3 -c "import sys; print(sys.stdout.encoding, sys.flags.utf8_mode)"
   # ascii 0

   Variable	What it does (docs.python.org)
   PYTHONUTF8=1	Turns UTF-8 mode on; 0 turns it off. The mode makes open() and paths use UTF-8
   PYTHONIOENCODING	Sets the encoding of stdin/stdout/stderr. On Windows it is ignored for an interactive console unless PYTHONLEGACYWINDOWSSTDIO is set; redirected files and pipes are not affected
   Python 3.15+	UTF-8 mode is the default (PEP 686, status "Final"). Before that, on Windows open() uses the system code page — set $env:PYTHONUTF8 = '1' before starting the interpreter

   For scp and rsync — file names are bytes, so the capture and transport rules above apply to whatever you call from PowerShell. ⚠️ We have not checked special settings for encoding their names themselves (rsync has an --iconv option — see man rsync, not verified here).

8. The safety rule before deleting

   🛑

   "Does not exist" often means "broken string"

   Never delete on the strength of a "missing" check if the path went through a pipe raw with non-ASCII. First prove you are reading the right string (decode from base64 or run the test from step 1), check existence and size, and only then act. Do the heavy path work on Linux (Python/bash) — UTF-8 is native there, and let Windows be only a thin transport.

   Situation	✅ Do	❌ Don't
   A Cyrillic path between Windows and Linux	base64 end to end	a raw string through the pipe
   Writing a file	-Encoding utf8 (or explicit UTF-8 without BOM for Linux)	-Encoding ascii, or "the default" in 5.1
   A script with Cyrillic for 5.1	UTF-8 with BOM, run with -File	a long string through -Command
   Accessing the path	-LiteralPath and a decoded string	an interpolated string with Cyrillic through a pipe
   Before deleting	a check with a decoded path	trusting "does not exist"

04Check

• You know which PowerShell you use and the values of $OutputEncoding and [Console]::OutputEncoding.
• The test from step 1 gives "equal: True" after the fix.
• Nothing on your side writes with -Encoding ascii; scripts with Cyrillic for 5.1 are UTF-8 with BOM.
• A Cyrillic path goes from Linux to Windows as base64 and is decoded to a Unicode string.
• Python on Windows runs with PYTHONUTF8=1 (or is 3.15+).

Quiz

05What's next

06Sources

1. about_Character_Encoding — default encodings and the BOM in 5.1 and 7, scripts without a BOM.
2. about_Preference_Variables — $OutputEncoding (ASCII in 5.1, UTF-8 in 7).
3. Console.OutputEncoding (.NET) · chcp — the console code page.
4. Python: command line and environment — PYTHONUTF8, PYTHONIOENCODING · UTF-8 mode · PEP 686 — the default mode from 3.15.
5. ssh_config (SendEnv) · sshd_config (AcceptEnv) — environment variables over ssh.