Non-ASCII text and parentheses in Windows batch files: commands cut into fragments after chcp 65001, and a ) inside echo closes the if block early
When cmd reads a batch file containing multibyte characters, later lines can be read from a shifted position, producing “is not recognized” errors with half a command name; chcp 65001, UTF-8 with BOM and GBK all drifted, and only pure ASCII was stable. Autarkia hit this three times, because the checker covered one directory and wasn’t in CI.
When cmd runs a .bat file containing multibyte characters (a Chinese comment, a Chinese echo), later lines can be read starting from a shifted position, and the errors show command names with their beginning chopped off. We compared variants: chcp 65001 at the top, UTF-8 with BOM, saving as GBK; all three still drifted, and only pure-ASCII content was stable regardless of code page. I couldn’t find a reliable source for why cmd miscalculates the position internally, so this only describes what we observed.
The first time was a launcher script. The errors looked like this:
'ain.py' is not recognized as an internal or external command,
'ASE_DIR' is not recognized as an internal or external command,main.py was read as ain.py and BASE_DIR as ASE_DIR, each missing its first character. Two more patterns surfaced at the same time, unrelated to encoding but equally capable of scrambling block structure:
if exist "%VENV%\Scripts\python.exe" (
echo Using venv (core)
rem ↑ 这个 ) 会把 if 块提前关掉，下一行跑到块外
:: 块内用 :: 当注释也会出问题，它其实是标签语法
)When parsing a parenthesized block, cmd doesn’t care that what follows echo is plain text: a ) ends the block. :: is really a label that nothing ever jumps to, and inside a block it breaks parsing; comments inside blocks need rem.
One script was a counterexample. In it, the ) of echo ... (%RETRY_COUNT%/%MAX_RETRIES%) happened to close the outer if block early, which put the following goto outside the block and delayed variable expansion to run time. Removing the parenthesis made the block whole, and then the variables expanded to nothing at parse time, giving goto was unexpected at this time. That script stayed an exception; the real fix is to move the retry loop out of the if block.
01Why it happened three times
After the first fix, six launcher scripts went pure ASCII, and a checker was added: ASCII only, no BOM, no parentheses in echo inside blocks, no :: inside blocks. Fed the pre-fix files, it caught all of them.
A month later, the root start.bat got a few Chinese comments and crashed on double-click with 'p_code_sync' is not recognized, code_sync.py cut in half. The checker didn’t catch it because it only scanned start_scripts/, and it was a manually run verification script, not part of CI. That investigation took a detour too: the exported file was CRLF so I suspected line endings, until I realised PowerShell’s Out-File had converted the output; the file in the repo was LF all along.
Scanning the repository while writing this, I found the third occurrence already on HEAD: a project-wide rename commit added a Chinese rem line to start_demo.bat, whose fourth line is chcp 65001. The same rename also turned “new variable name first, old name for compatibility” into the same name twice, so the compatibility fallback is gone without anyone noticing.
Remembering the rule doesn’t help; it gets broken by someone else, by a bulk rename, by me three months later. The fix I’ve noted for the main project is to make the checker scan git ls-files '*.bat', run it as a normal unit test in CI, and add a .gitattributes line for consistent line endings:
*.bat text eol=crlfThe repo’s .bat files are currently a mix of CRLF and LF. LF-terminated batch files have known risks around goto and call :label; the drift here wasn’t caused by that (the pure-ASCII LF versions ran fine), so unifying on CRLF is just prevention.