Canary

Try it

Mint a key, then check the sample log that appears. The key that verifies tokens is created and kept in this browser only. Nothing is sent anywhere.

Why use it

Most leaks are found late, if at all. Data goes into a model prompt, a log line or a shared document, and months later nobody can say which path it took. A real secret that leaks is also a real risk, so you cannot safely use one as bait.

A canary is bait that is safe to lose. It is a credential-shaped string that does nothing, planted somewhere that only a leak would carry it. When it appears where it should not, you have proof, and the tag tells you where it started.

Good places to plant one

• A documents folder that an AI assistant can read but must never quote. Scan its replies for the canary.
• A staging config or a private repo, to see whether it ends up in logs or in a public paste.
• A system prompt, to test whether your app can be tricked into revealing it.
• A seeded record in a test database that you export to a vendor.

How it works

1. 1

   You pick a tag and a secret

   The tag is four letters that say where you plant it, such as PROD. The secret key is a string only you know.
2. 2

   Canary builds a token

   The token has the right shape for the style you chose, with your tag, random characters, and a short check code computed from your secret key.
3. 3

   You plant it

   Put it in a file, a document or a prompt. Canary stores nothing, so keep a note of which tag went where.
4. 4

   You scan text later

   Run any text through the checker. It finds token-shaped strings and recomputes the check code. If it matches your key, it is yours, and the tag is read back out.

Set it up

You need Node 22.18 or newer.

git clone https://github.com/tcvdh/leakybyte.git
cd leakybyte
npm install

# a long secret that only you know. Keep it in your secret manager.
export LEAKYBYTE_CANARY_KEY="paste-a-long-random-string-here"

Mint

The arguments are a tag, then a style: anthropic or aws. Real output:

$ npm run lb canary mint prod anthropic
sk-ant-api03-PRODQ43CBT4QZDICOPT4A3QC34AIRD

$ npm run lb canary mint ci aws
AKIACIXXA6YXAAR3PRFT

Check

Text goes in on stdin. Exit code 0 means no canaries, and 2 means at least one was found, so it works as a gate in scripts and CI.

$ cat app.log | npm run lb canary check
LEAK: anthropic canary "PROD" at char 412
$ echo $?
2

Examples

A decoy in a staging file

# staging.env : a decoy that real code never reads
ANTHROPIC_API_KEY=sk-ant-api03-PRODQ43CBT4QZDICOPT4A3QC34AIRD

No real code reads this file. If the token shows up in a log, an error report or a pasted snippet, something read the file that should not have.

A gate in CI

Run your assistant against a test prompt that tries to extract the planted token, then scan what it wrote.

name: canary-check
on: [push]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm ci
      # exits with code 2, and fails the job, if a canary appears in the output
      - run: cat model-output.txt | npm run lb canary check
        env:
          LEAKYBYTE_CANARY_KEY: ${{ secrets.LEAKYBYTE_CANARY_KEY }}

Token formats

Characters are uppercase A to Z and 2 to 7. A tag is cleaned the same way: uppercased, anything else dropped, and padded with X to four letters, so ci becomes CIXX.

Limits to know about

• The check code is 4 characters, about 20 bits. It prevents accidental matches. It does not stop a determined person who knows your format from forging a token that looks verified.
• Canary finds a token only if it appears in text you scan. It does not watch the network.
• Two styles exist today. More formats are on the list.
• The CLI reads the key from an environment variable. The browser demo keeps its own key in local storage.