Using LLMs to do security analysis at the git diff level — what works, what doesn't, and why structured output matters

I've been experimenting with piping raw git diff output into LLMs for automated security review, and I wanted to share what I've learned because some of the results surprised me. The problem that started this A teammate refactored a SQL query from string concatenation to an f-string. The diff looked like an improvement: - query = "SELECT * FROM users WHERE id = " + user_id + query = f"SELECT * FROM users WHERE id = {user_id}" Three reviewers approved it. It looked cleaner. But the vulnerability was identical — both are injection vectors. The cosmetic improvement actually made it harder to catch because it looked like the dev was modernizing the code. This is the kind of thing that made me think: can an LLM reliably detect that a diff looks like a fix but isn't? The architecture I built a FastAPI service that accepts a raw diff string and returns structured JSON with severity-classified security and quality issues. Here are the key design decisions I made and why. Why diffs and not full