Ch 19 / 24 Security, Shifted Left 0/0 exercises Exercises ↓

AmouAI Hub/Courses/Software Engineering for AI Engineers/Chapter 19

Part 4 · Making Systems Secure and Reliable · Chapter 19

Security, Shifted Left

You do not need to become a security specialist. You do need to be the person who notices, at the moment the code is written, that this endpoint takes an id from the user and never checks who owns it — because that particular defect is invisible to every scanner you can buy, and it is the one that keeps happening.

Reading
OWASP Top 10 · STRIDE
Focus
Ownership, not scanning
Modes
JavaScript · choice
Exercises
12

By the end of this chapter you can

  1. Threat model a flow in fifteen minutes with four questions
  2. Parameterise every query, including the ones that look like exceptions
  3. Write an ownership check that fails closed
  4. Validate a user-supplied URL against private ranges and redirects
  5. Explain why broken access control is the class scanners miss
  6. Keep secrets out of logs, build output and error messages

1You are partly a security engineer now

Not because anyone appointed you, but because the decisions that matter are made in the code and cannot be inspected back in afterwards.

A security team can run scanners, set policy, and review a design. What they cannot do is notice that the getInvoice function takes an id and returns whatever it finds — because that code looks exactly like correct code, and the missing thing is a line that was never written. The people who can catch it are the person who wrote it and the person who reviewed it.

The good news is that the vast majority of real vulnerabilities are a small number of shapes, and knowing those shapes is a few hours of study rather than a career. You are not being asked to find novel classes of attack; you are being asked to recognise five or six patterns when they appear in a diff.

The framing that makes this manageable: every input from outside your system is hostile until it has been through something that makes it safe — and “outside” includes another team’s service, a webhook payload, a filename, a URL you were asked to fetch, and text a model produced from content you did not write.

The idea to keep

Ask two questions of every handler: who is allowed to do this, and where does this value come from. Almost every vulnerability in this chapter is one of those two questions going unasked.

2Threat modelling in fifteen minutes

Threat modelling has a reputation for being a two-day workshop. The version that fits in a design review is four questions, and it catches most of what the two-day version catches.

  1. What are we building? Draw the flow. Boxes and arrows, five minutes, on a whiteboard.
  2. What can go wrong? Walk each arrow and ask the STRIDE questions — can someone pretend to be someone else, change data in flight, deny having done it, read what they should not, take the system down, or gain more privilege than they had?
  3. What are we going to do about it? Per finding: mitigate, accept with a reason, or transfer.
  4. Did we do it? The step that gets skipped, and the one that makes the exercise real.

Two practical notes. Do it on the flow rather than the system — “a user exports their invoices” is a tractable subject and “Ledger” is not. And write the findings down where the code is, because a threat model in a document nobody opens has the same fate as the diagram in Chapter 16.

3The surface that keeps appearing

Four classes account for most of what actually happens to ordinary applications. None is exotic and all four are cheap to prevent at the moment the code is written.

ClassThe shapeThe fix, stated once
InjectionUser data is concatenated into something that gets interpreted — SQL, a shell command, a template, HTMLNever build the instruction from the data. Parameterise, escape on output, and pass arguments as an array rather than a string
Broken access controlThe system checks who you are and forgets to check what you may touchScope every query by the actor, in one enforced place. Section four is entirely about this
SSRFYour server fetches a URL the user chose, from inside your networkResolve, validate against private ranges, allow only expected schemes and ports, re-check on redirect
SecretsA credential reaches somewhere it can be read — a log, a build artefact, an error page, a repositoryNever in code; redact at the logging boundary; rotate. Section five

Injection deserves one clarification because the standard advice gets misapplied. The fix is not “escape the input” — it is never construct the instruction out of the data. A parameterised query sends the SQL and the values over separate channels, so no value can become syntax; an escaping function tries to neutralise syntax inside a single channel, and it is a rule with exceptions you will eventually meet. The same distinction explains why execFile(cmd, [args]) is safe where exec(cmd + ' ' + arg) is not.

And the awkward cases are where this breaks down in practice. You cannot parameterise a column name in an ORDER BY, which is exactly why somebody concatenates it. The answer is an allowlist — map the user’s sort=due_date to a known column through a lookup table, and reject anything not in it — and the reason this matters is that the awkward case is where the vulnerability always is.

4Why broken access control is the one scanners miss

A scanner can find a SQL string built by concatenation, because that is a syntactic pattern. It cannot find a missing ownership check, because the correct code and the vulnerable code look identical.

Consider two handlers. One returns an invoice by id. The other returns an invoice by id after checking that it belongs to the caller’s organisation. The difference is one predicate, and no tool that has not been told your authorisation model can tell which one is right — because for a public endpoint, the first one is right.

This class has a name — insecure direct object reference, or IDOR — and it is consistently near the top of every vulnerability survey, for a reason worth internalising: it is a missing thing, and static analysis is much better at finding present things than absent ones.

Three defences, in increasing order of durability:

DefenceWhat it doesWhy it is not enough on its own
Check in the handlerif (invoice.orgId !== session.orgId) throwCorrect where it appears, and it has to appear in every handler, forever, including the one written next Tuesday
Scope in the repositoryEvery query goes through a method that takes the actor and adds the predicateMuch better — the check cannot be forgotten because it is not a step. It can still be bypassed by writing a raw query
Enforce in the databaseRow-level security, so the predicate is applied by the engineThe strongest, and it costs operational complexity and a per-request session variable

For most teams the repository layer is the right answer: it removes the possibility of forgetting, it is testable against a database containing two organisations, and it puts the security-relevant code in one file somebody can review. What makes it work is the discipline that nothing queries the tables directly — which is a rule the build can enforce, exactly as in Chapter 13.

Two more properties are worth naming because they are cheap and frequently missed. Fail closed: if the actor is missing, unknown, or the check throws, the answer is no — a check written as if (user && user.orgId !== target) throw passes silently when user is undefined, which is precisely the case an attacker will produce. And return the same answer for “does not exist” and “not yours”: a 404 for both, because a 403 tells the attacker the id was real, which is how enumeration works.

5Secrets: where they end up, and how to stop it

Secrets rarely leak from the place they are stored. They leak from the six places they are copied to on the way.

Where it ends upHowThe control
Version controlA key committed “temporarily”A pre-commit scanner, and treat any committed secret as burned — rotate, do not just remove the commit
LogsLogging the whole request object, headers includedRedact at the logging boundary, by key name, so nobody has to remember at each call site
Error pages and stack tracesA connection string in an exception messageGeneric errors to the client; details to the log, redacted
Build outputA CI step echoing its environmentMasked variables, and no set -x in a step that touches secrets
Client bundlesAn environment variable inlined at build timeKnow which prefix your framework exposes to the browser, and check the bundle
A model promptA key in context that gets echoed backRedact before the call. Anything in a prompt may appear in an output

The redaction one is worth doing properly because it is the highest-volume leak. Redacting at each call site fails the first time somebody logs an object they did not construct; redacting in the logger, by key name, against a list including password, token, secret, authorization, api_key and cookie, catches the ones nobody thought about. And it should preserve the shape — replace the value, keep the key — because a log line that silently drops fields is a debugging problem you have traded for a security one.

One habit worth more than any tool: treat a leaked secret as burned. Removing the commit does not help, because the repository was cloned, the CI cached it, and the value was in a log. The only response that works is rotation, and a team that can rotate a credential in ten minutes handles this as an inconvenience rather than an incident.

6Build: threat model one Ledger flow

Fifteen minutes on the export flow, written out, so the shape is concrete.

The flow. A user clicks Export. The API queues a job. A worker reads the organisation’s invoices, writes a CSV to object storage, and POSTs a notification to the integrator’s webhook URL. The user downloads the file from a signed URL.

BoundaryThe threat that mattersThe controlVerified by
Browser → APIA request supplying its own orgIdTenant comes from the session, never the requestA test asserting a request with a foreign orgId returns the caller’s data, not the foreign one
API → queueUnbounded exports as a denial of serviceRate limit plus one running export per organisationA test that a second export returns the running one
Worker → databaseThe tenant predicate missing on the export queryRepository scoping, enforced by the buildAn integration test with two organisations in the database
Worker → object storageA predictable object key readable by anyoneRandom key, private bucket, signed URL with a one-hour expiryA test that the unsigned URL returns 403
Worker → webhookSSRF to an internal addressResolve, reject private ranges, https on 443, re-check on redirect, no response body storedTests for a private address, a hostname resolving to one, and a redirect to one
Signed URL → userThe link shared or leaked after the factShort expiry, one-time use, audit entry on accessA test that a second fetch of the same URL fails

The last column is what makes this a piece of engineering rather than a document. Every control has a test named for the threat it prevents — which is Chapter 17’s point applied to security, and Chapter 16’s point applied to a decision nobody should have to rediscover.

Notice also how much of this table is the same two questions from section one. Four of the six rows are “who is allowed to do this”; one is “where did this value come from”; and one is rate limiting. That distribution is typical, and it is why access control deserves more of your attention than the exotic categories.

Checkpoint

Playground

A second flow — the AI drafting path, where the untrusted input is text rather than a parameter. Walk the arrows and notice that the categories are the same ones.

Exercise set

Twelve problems, most of them building the specific controls this chapter argues for. The tests are adversarial: they send the inputs an attacker would send, not the ones a user would. Your work is saved in this browser.

All Warm-up Core Challenge Reset chapter

Chapter 20 — The AI Supply Chain

You execute far more third-party code than your own, and an agent now chooses some of it. Next chapter closes Part 4: package hallucination, reviewing what was added rather than what was written, and prompt injection when the agent holds real tools.

Continue →