The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Cenk Kurtoğlu reports finding more than 60 live Supabase service_role keys while scanning public GitHub repositories over three days. He then tested whether models could classify supplied examples containing secret-like credentials. That small benchmark measured snippet triage—not whether an AI can independently search a repository and discover secrets.
What the benchmark tested—and what it did not
Kurtoğlu’s September 30, 2026 article describes two related but distinct tasks. In the first, a system must search a codebase, choose files or tools to inspect, and find exposed credentials. In the second, it is handed a snippet and asked to identify what is in it. The benchmark tested the second task.
As an Amazon Associate I earn from qualifying purchases.
The 60-plus live-key count is the author’s account of a three-day scan, not a representative measurement of how often public repositories contain exposed credentials. The article describes examples he says he encountered, including hardcoded PHP configuration, real credentials in .env.example, a service_role value in a NEXT_PUBLIC_ variable, Docker Compose files containing multiple secrets, and deployment documentation with credentials. Those examples and the count have not been independently audited here.
For the benchmark, the author built 10 snippets modeled on those patterns. Every credential in the test was fake but structurally valid: the exercise did not publish or test the live keys from the scan. Models were asked to return strict JSON with has_service_role, has_anon, has_db_password, warning_level, and reasoning. A case counted as correct only when every asserted field matched the expected answer; the reported score was the share of cases fully correct.
#1 Best Overall
- POWERFUL SECURITY KEY: The Security Key C NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key C NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key C NFC via USB-C and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
The 10 test cases
- Hardcoded keys.
- An
.envfile containing only an anon key. - A client-side anon-key fallback.
- Deployment documentation containing credentials.
- Placeholders intended to expose false alarms.
- A
service_rolekey in a client-prefixed variable. - A real key in
.env.example. - A commented-out key.
- Two keys together.
- A base64-obfuscated
service_rolekey.
The author says temperature was set to zero and each participant could submit once. These choices describe this particular run; they do not make its scores a general guarantee of model behavior.
What the author reported
The figures below are Cenk Kurtoğlu’s results from that benchmark, not independently reproduced scores or durable rankings. A perfect score means all 10 test cases were fully correct under the author’s rubric.
Rank #2
- POWERFUL SECURITY KEY: The YubiKey 5C NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5C NFC secures 100+ of your favorite accounts, including email, password managers, and more
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5C NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
- PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts
| Model | Reported result | What the article says |
|---|---|---|
| Claude Sonnet 5 | 10/10 | Described as a clean sweep. |
| Gemini 3.7 Flash | 10/10 | Described as a clean sweep. |
| Gemini 3 Flash Preview | 10/10 | The author says it decoded the base64 case. |
| Gemini 3.1 Flash Lite | 10/10 | Described as a clean sweep. |
| GPT-5.4-nano | 8/10 | Missed the commented-out secret and the base64-obfuscated key. |
| DeepSeek-R1-0528 | 0/10 | The author says it marked every case critical, including placeholders. |
| Qwen3-Next-80B | Partial; no completed score | Passed five of six assertions before rate-limiting interrupted the run. This is not a five-of-six-case score. |
| GPT-OSS-120B | Excluded; no score | Repeated provider errors under load prevented a completed run. |
The article says nine models were involved overall, but the results described above do not amount to nine comparable completed scores: Qwen’s run was interrupted and GPT-OSS-120B was excluded. The useful result is the contrast within this particular exercise: some models handled its cases, while another produced a broad false alarm and GPT-5.4-nano missed two less-obvious examples.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsWhy identifying the key type matters
A credential’s presence is not enough to determine its risk. Supabase distinguishes elevated server-side credentials from keys intended to be public, and the benchmark’s labels should be read in that context.
Rank #3
- POWERFUL SECURITY KEY: The YubiKey 5 NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 NFC secures 100+ of your favorite accounts, including email, password managers, and more
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
- PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts
Secret and legacy service_role keys
Supabase documents that a secret key authorizes access through the service_role Postgres role, which has the BYPASSRLS attribute. The older service_role key is also elevated and JWT-based; Supabase recommends using newer secret keys where possible. These credentials belong only in controlled backend components—not browser code, shipped client packages, public documents, or source control. As Supabase puts it in its API keys documentation: “Never use a secret key in the browser or expose it to customers.”
Publishable and legacy anon keys
Publishable keys—and legacy anon keys—are designed for public or client-side use, provided database grants and row-level security (RLS) are configured appropriately. Their public status does not make every database operation public: grants determine which operations a role may perform, while RLS policies constrain rows for roles subject to RLS. Supabase’s RLS guidance is important here: RLS does not replace correct grants, and it does not constrain the elevated service_role role that bypasses it.
Rank #4
- POWERFUL SECURITY KEY: The Security Key NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key NFC via USB-A and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Supabase says legacy anon and service_role keys are being deprecated by the end of 2026 in favor of publishable and secret keys. New and legacy keys can coexist during migration; creating a replacement does not itself revoke the old credential.
What these results can—and cannot—tell you
The benchmark supports a narrow conclusion: on 10 supplied, fake-key examples, model outputs differed, and the author’s rubric made both edge cases and false positives consequential. DeepSeek-R1-0528’s reported treatment of placeholders illustrates why a detector that flags everything can be less useful than its raw alert count suggests.
Best Value
- POWERFUL SECURITY KEY: The YubiKey 5 is a versatile physical passkey that protects your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 secures 100+ of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 via USB and tap it to authenticate. No batteries, no internet connection, and no extra fees required.
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
It does not establish how prevalent exposed Supabase keys are across public repositories, how well models perform on unseen code, or which model will be most reliable in another run. Nor does it show that a model can autonomously locate secrets across a multi-file repository. The article’s question, “does performance drop when the secret is two hops away?”, is a proposed next test, not a result from this benchmark.
Useful next evaluations would separate completed-case accuracy from precision on placeholders and recall on comments or obfuscated values. They could also test whether relevant evidence spread across files can be found, whether a model actively searches with tools, and whether its remediation advice sequences replacement and retirement correctly. The last three are future evaluation questions, not capabilities established by these scores.
If you find a leaked Supabase key
Follow Supabase’s documented, key-type-specific rotation guidance rather than assuming every credential can be revoked instantly in the same way. The sequence is to remove the cause of exposure, create a replacement, deploy it to all consumers, verify they use the new credential, and only then retire or deactivate the compromised one as appropriate for its key type.
Recommended Free Tools
- Fix the exposure. Remove the credential from the affected code or configuration and address how it entered a public location. Do not treat removing a visible copy as a substitute for replacing a credential that may already have been copied.
- Create a replacement. Use the appropriate current Supabase key type and follow the documented procedure for that type.
- Update every consumer. Deploy the replacement anywhere the old credential is used, including relevant backend services and deployment configuration.
- Verify the change. Confirm all components are operating with the replacement before retiring the compromised credential.
- Retire or deactivate the old key. Use the procedure Supabase documents for that key type. Creating a new key alone does not disable the old one, and rotation behavior differs by type.
For publishable or legacy anon keys, investigate whether the associated grants and RLS policies permit unintended access; public exposure alone is not equivalent to a secret-key leak. For an exposed secret or legacy service_role key, treat it as an elevated backend credential and work through the replacement and retirement sequence promptly.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
