Japanese Encoding Lookup & Mojibake Fixer

Why Japanese text turns into mojibake, and what can be recovered

Japanese has had more competing text encodings than almost any other language, and mojibake (文字化け, "character transformation") is the word for what happens when text written in one is read as another. Three legacy encodings did most of the damage. Shift_JIS was designed so that Japanese could live alongside the single-byte katakana of early computers, and it became the default on Windows. EUC-JP fitted Japanese into the conventions of Unix systems. ISO-2022-JP, with escape sequences that switch between ASCII and Japanese, was the standard for email. All three encode the same core character set, JIS X 0208, but they arrange its bytes differently — so a file saved in one and opened in another becomes nonsense.

Every mix-up leaves a signature

The garbage is not random. UTF-8 read as Windows-1252 turns 文字 into strings like 文字 — accented Latin letters in threes, one per UTF-8 byte. UTF-8 read as Shift_JIS produces unfamiliar kanji broken up by � marks. EUC-JP read as Shift_JIS comes out as a soup of half-width katakana, because most EUC-JP bytes happen to fall in Shift_JIS's half-width range. This tool uses those signatures: it re-creates the bytes the wrong decoder must have seen, decodes them the right way, and ranks the results by how much they read like Japanese — hiragana and common kanji count in favour, the debris of a wrong decoding counts against.

Some mojibake can be undone exactly and some cannot. When a decoder has already replaced bytes with �, those bytes are gone, and an honest repair shows the gap instead of guessing what filled it.

Recoverable and unrecoverable

Whether a repair is possible depends on the direction of the mistake. UTF-8 shown as Windows-1252, or Shift_JIS shown as Windows-1252, is lossless: every byte survives as some Latin character, and the original comes back perfectly. UTF-8 shown as Shift_JIS is partly destructive — some byte pairs land on unassigned codes and are replaced with �, taking a character on each side with them. Encode 文字化け in UTF-8, read it as Shift_JIS, and only 化 can be recovered; the tool shows □化□ rather than pretending. Shift_JIS shown as UTF-8 is the worst case: nearly every byte becomes �. If you still have the original file, reopening it with the right encoding always beats repairing the damaged copy.

The characters that break even when nothing is mixed up

A few characters misbehave inside a single encoding. Browsers and Windows read the Shift_JIS byte 5C as a backslash, but Japanese fonts draw it as a yen sign — which is why Windows paths appear as C:¥ in Japan — and the web standard writes the yen sign itself as that same byte, so a ¥ sent out comes back as \. Not every decoder reads 5C that way: Python's shift_jis_2004 and macOS iconv's SHIFT_JIS return ¥.

The wave dash 〜 (U+301C) depends on the table. The WHATWG Encoding Standard that browsers follow maps Shift_JIS 81 60 to the full-width tilde ～ (U+FF5E), exactly as Microsoft's CP932 table does, and neither has a code for 〜. Unicode's own JIS X 0208 mapping, and converters that follow JIS, map 81 60 to 〜 instead. The same two bytes come back as different characters depending on which program reads them, and that disagreement is the "wave dash problem".

Characters such as ① and 髙 are vendor additions rather than part of JIS X 0208: the kind Japanese software calls 機種依存文字 ("machine-dependent characters") or 環境依存文字.

One name, several tables

"Shift_JIS" does not name a single table. On 2 Oct 2026 we decoded every two-byte code the browser defines with other converters. Microsoft's CP932 table agreed with the browser on every one. Unicode's JIS X 0208 mapping disagreed at seven positions, the wave dash among them. A strict Shift_JIS converter (Python's shift_jis, macOS iconv's SHIFT_JIS) rejected all 845 vendor codes. And a JIS X 0213 converter (Python's shift_jis_2004, macOS iconv's SHIFT_JISX0213) read all 373 characters of IBM's extension set (360 of them kanji) as other kanji, with no error: 髙 (FB FC) became 騠. Those are the IBM codes FA 40 to FC 4B that a browser writes; the NEC-selected copies of the same characters (lead bytes ED and EE) were misread too, so the result holds for both forms. A file can be "Shift_JIS" to the program that wrote it and to the program that read it, and still change on the way. When you convert a file written on Windows, name the table: CP932, also called Windows-31J.