Skip to content

Token & Whitespace Inspector

Show a prompt's token boundaries as a heatmap and flag zero-width, non-breaking and bidirectional control characters that change how a model reads it.

Token & Whitespace Inspector

Loading tokenizer… showing estimate

about 14 tokens, 2 hidden characters

Characters
54
Tokens (est.)
~14
Whitespace
9
Hidden chars
2

Token heatmap (estimate — tokenizer loading)

Summarize·theNBreportZW·in·3·bullet·points. → Be·concise.

2 hidden / risky characters found

  • nbsppos 13U+00A0No-Break Space
  • zero-widthpos 20U+200BZero Width Space

Bidi controls and zero-width characters can hide instructions or spoof code (the "Trojan Source" class of attacks). Remove them unless you added them intentionally.

Updated . Provided as is. Check the output before you rely on it in production.

How to use Token & Whitespace Inspector

  1. 1

    Paste your prompt

    Drop the prompt or text you want to inspect into the input box. The heatmap and stats update as you type.

  2. 2

    Read the token heatmap

    Each token gets an alternating background so you can see exactly where the tokenizer splits your text, with a live token count.

  3. 3

    Reveal whitespace

    Toggle whitespace markers to turn spaces into middots and surface tabs and newlines, exposing trailing or doubled spacing.

  4. 4

    Reveal hidden characters

    Toggle hidden-character highlighting to catch zero-width, non-breaking, and bidi controls, each labeled with its codepoint and name.

  5. 5

    Copy the report

    Copy a plain-text summary of character, token, whitespace, and hidden-character counts plus the position of each flagged character.

Questions and answers

What does this tool show?
Tokens are the chunks of text a model actually reads, and invisible characters can change how text splits into them. This inspector colors each o200k token in your prompt, counts characters and tokens, and flags zero-width characters, non-breaking spaces and bidirectional controls that can alter or hide instructions.
Which tokenizer does it use?
It uses the o200k byte-pair tokenizer (OpenAI's current encoding) loaded on demand in your browser. Until it loads, token counts fall back to a fast characters-divided-by-four estimate.
What hidden characters does it detect?
Zero-width characters (U+200B-200D, U+FEFF), non-breaking spaces, and bidirectional control characters (U+202A-202E, U+2066-2069) that enable Trojan Source style hidden-instruction attacks.
Why do tokens matter for prompts?
Token count drives cost and context limits, and odd whitespace can split words into extra tokens or shift meaning. Seeing the boundaries helps you trim and clean a prompt.
Is my prompt sent to a server?
No. Tokenization and character analysis run entirely in your browser. Your prompt is never uploaded or stored.
For AI agents: how to call this tool

Machine-readable contract, endpoints and examples. Humans can ignore this section.

Best Path For Builders

Browser workflow

Runs instantly in the browser with private local processing and copy/export-ready output.

Browser Workflow

This tool is optimized for instant in-browser execution with local data handling. Run it here and copy/export the output directly.

/token-whitespace-inspector/

For automation planning, fetch the canonical contract at /api/tool/token-whitespace-inspector.json.