What does an automated security scanner actually check?

A scanner compares your app against a library of known-bad signatures: dependencies with published vulnerabilities, missing security headers, protections that are switched off entirely, key strings matching known secret formats. These are all presence-or-absence questions with a universal right answer, which is exactly what pattern-matching is good at. What no scanner holds is a description of your business's rules — who may see which record, who may change which field — so it cannot judge whether an access rule that is present is also correct.

How can an app pass a security scan and still be insecure?

Because scanners test for the presence of a control, not its correctness. A row-level security policy that permits every user is present, syntactically valid, and passes; a policy that is missing entirely fails. The same asymmetry covers endpoints that check you are signed in but never check the record is yours, webhook handlers that skip signature verification, and APIs that return far more fields than the page displays. AI code generators reliably emit the shape of a control — an auth middleware on every route, a policy on every table — so a generated app can score well precisely where its logic is weakest.

What security checks can I run myself without hiring anyone?

Six, each about five minutes, all against your own app: create two test accounts and try to read one account's record while signed in as the other; open a private page while signed out and watch whether data arrives anyway; compare an API response in the browser's network tab against what the page actually displays; search the delivered JavaScript bundle for secret-key prefixes; run a history-wide secret scanner such as gitleaks or trufflehog over the whole repository; and check that every webhook handler verifies the provider's signature before acting on the payload. A failure on any of these is a real finding, not a maybe.

Should I still run automated security scans at all?

Yes. Scanners are the cheap, continuous layer: they catch the crude cases — a protection switched off, a dependency with a published vulnerability, an obvious leaked key — immediately and repeatedly, which no human review can do economically. The mistake is not running a scanner; it is reading a clean scan as an answer to a question it never asked. Keep the scanner, and treat its output as a floor rather than a verdict.

Your AI-Built App Passed the Security Scan. That Is Not the Same as Being Secure (2026)

Q: What does a manual security review add beyond the scan?

Judgment about intent. A reviewer reads each access rule against the roles it is meant to separate, traces the two or three riskiest flows end to end — sign-up and role assignment, anything touching money or other people's data — and checks that what the code enforces matches what the business means. None of that is automatable, because the intended rules exist in your head and your terms of service, not in any signature database a scanner can match against.

The short version

A scanner tests whether a control is present, not whether it is correct. An access rule that permits everyone is present, and passes. A rule that is missing entirely fails. That asymmetry is the whole story.
Generated apps score well for a structural reason. AI generators reliably emit the shape of every control — a policy on every table, an auth check on every route — and shape is all a scanner inspects.
The defects that matter live in the logic: who a rule actually admits, whose record an endpoint actually returns, whether a webhook believes anyone who posts to it.
Most of them are checkable by a non-specialist in about five minutes each. The checklist below is the useful part of this page.

What a scanner sees — and what it cannot

An automated scanner, whatever badge it carries, works one way: it compares your app against a library of known-bad signatures. A dependency version with a published vulnerability. A missing security header. A protection that is switched off entirely. A string in the code matching the format of a known secret key. Every one of these is a presence-or-absence question with a universal right answer, and pattern-matching answers them quickly, cheaply and repeatedly. That is real value, and I am not going to pretend otherwise — run the scanner.

Now consider what is not on that list. "Only a project's members may read its documents." "Only staff may change an order's status." "A user may edit their own profile and nobody else's." These are the rules that make your app yours, and they exist nowhere a scanner can look — not in a signature database, not in the code as a checkable assertion, only in your head and your terms of service. A scanner cannot verify that an access rule matches your intent, because it does not know your intent.

Which produces the asymmetry this whole page turns on: a missing control fails the scan, but a wrong control passes it. A database policy that reads, in effect, "allow everyone" is present, enabled and syntactically valid. Green tick. It also hands every row to anyone who asks.

Why AI-built apps pass anyway

Code generators are trained on millions of applications, so they are remarkably good at producing the outward form of security: every route gets an authentication middleware, every table gets a policy, every config gets its headers. The form is what a scanner checks, the form is all there, and the score is high. Whether the middleware actually restricts anything, and whether the policy separates the roles you need separated, is the part generators get wrong most often — and the part no scanner inspects.

This is not a hypothetical failure mode. In 2025 a researcher disclosed CVE-2025-48757: Lovable-generated apps shipping with Row Level Security missing or wrong, with a confirmed count of 170+ live applications whose email addresses, messages, payment records and API keys were readable by anyone holding the public anon key (Superblocks, TNW). The scanner the platform shipped in response checks whether RLS is enabled, not whether a policy is correct — a policy of USING (true) is enabled, passes, and returns every row in the table. Nor is this one platform's carelessness: Veracode's 2025 study across 100+ models found AI-generated code chose the insecure implementation about 45% of the time, with no improvement in newer models (Veracode).

So a generated app and a scanner are a specifically bad pairing: the generator's most reliable output is exactly the thing the scanner measures, and the generator's least reliable output is exactly the thing the scanner cannot see.

The defect classes that survive a clean scan

1 — Authorisation that trusts client-supplied identifiers

The request says "fetch record 4821" and the server fetches record 4821, checking only that the requester is signed in — never that record 4821 belongs to them. Change the number, read someone else's record. OWASP calls this broken object level authorization and ranks it the number-one API security risk (OWASP API Security Top 10). To a scanner the endpoint looks fine: authentication is present. Whose record comes back is invisible to it.

2 — Permissive row-level security

The database enforces per-row access through policies (Supabase docs), and generators dutifully create one per table. The failure is a policy scoped to "any signed-in user" where it should say "the row's owner" — or the blunt USING (true). Supabase's own database advisors catch the crude cases, RLS disabled or enabled with no policy at all; they do not judge whether the policy that exists is right, and nothing automated currently does.

3 — Secrets reachable from the browser

Anything compiled into your frontend bundle is delivered to every visitor — that is what a frontend is. Publishable keys are designed for this; admin, service-role and secret keys are not, and a generator that needed a privileged call to work will sometimes put the privileged key where the call was: in client code. Repository scanners look for known key formats in the code they are shown; a key pasted into a hosting dashboard's environment settings and injected at build time can reach the bundle without ever appearing in the repository at all.

4 — Endpoints that authenticate but never authorise

Authentication asks "who are you"; authorisation asks "and are you allowed". They are separate steps, and generated code regularly ships the first without the second — an admin endpoint any signed-in user can call, an update route that accepts changes to fields the user should never touch. Broken access control tops OWASP's list of web application risks (OWASP Top 10, A01), and it is precisely the class a scanner cannot detect: the check that is missing was never a pattern in the first place.

5 — Webhooks that believe anyone

A webhook is an endpoint that accepts instructions from the outside world — "this payment succeeded", "this subscription was cancelled" — and acts on them. Providers sign their deliveries so your handler can verify the message really came from them; both Stripe and GitHub document signature verification as required practice. A handler that skips the verification and trusts the payload will act on anything sent to its URL. From the scanner's side there is nothing to see: the endpoint exists, responds, and looks like every other endpoint.

6 — APIs that return more than the page shows

The page displays a name and an avatar; the API response behind it carries the full database row — email address, phone number, internal flags, sometimes tokens. Everything in that response is available to anyone who opens their browser's developer tools, whatever the page chooses to render. OWASP tracks this as broken object property level authorization (OWASP API Security Top 10). Generators produce it constantly, because returning the whole object is the path of least resistance — and no scanner knows which fields your page meant to show.

The checks you can run yourself

Everything below runs against your own app, needs no tools beyond a browser and two test accounts you create yourself, and takes about five minutes per check. This is the part of the page I would ask you to keep even if you close the tab afterwards.

The two-account test. Create two accounts of your own. Signed in as the first, open a record only that account should see, and note its identifier in the address bar or the network tab. Sign in as the second account in a private window and request that same identifier. Pass: refused, or not found. Fail: the second account reads the first account's data. Repeat for the most sensitive record type you hold.
The signed-out test. Copy the address of a page showing private data, sign out, and open it in a private window with the network tab open. Pass: you land on the login screen and no data arrived in any response along the way. Fail: the page redirects but a response in the network tab quietly contained the data anyway.
The response-body test. Open developer tools, load a page that lists other users or shared content, and read the actual API response. Compare it to what the page displays. Fail: email addresses, phone numbers, tokens or internal fields in the response that the page never shows.
The bundle test. In developer tools, search the JavaScript your app delivers for your providers' secret-key prefixes — sk_live, service_role — and for the word secret. Anything found in a file the browser downloaded is public, whoever it was meant for.
The history test. Run a history-wide secret scanner such as gitleaks or trufflehog over your whole repository, not the latest commit. A key committed early and "removed" later is still in the history, and still live until rotated.
The webhook question. For each webhook handler in your code, check that the first thing it does is verify the provider's signature — the Stripe and GitHub pages linked above show exactly what that looks like. If you cannot tell whether it does, that is itself the finding to hand a developer.

A failure on any of these is not a maybe — it is a real defect, and each one has a specific, bounded fix. A clean sweep is meaningful too: it says the most common classes of generated-app failure are absent from the paths you tested.

What a review adds beyond the scan

Keep the scanner. It is the cheap, continuous layer, and it catches the crude cases the moment they appear — which a human review cannot do economically. What a review adds is the layer the scanner is structurally blind to: a person reads each access rule against the roles it is meant to separate, traces the two or three riskiest flows end to end — sign-up and role assignment, anything touching money or other people's data — and checks that what the code enforces is what the business means. The checklist above samples that work; a review does it exhaustively, policy by policy, and writes down what it checked so you can see what was and was not covered.

If you want help

You may not need any. If the six checks pass, your app holds nothing especially sensitive, and you rerun them after each significant generated change, you are ahead of most teams at this stage and can reasonably stop there. Where I would want a review is when a check fails and you are not sure how far the problem extends, or when you hold other people's data and want the access rules actually read rather than sampled. That is what Appaya's review is: the work above, done exhaustively against your real repository, with a report that says plainly what was checked — and says so if everything is fine where it is.

Request a fit check How the audit works

Your AI-built app passed the security scan. That is not the same as being secure