AI coding assistants speed up boilerplate, but they also autocomplete insecure patterns. Security researchers have measured how often GitHub Copilot suggests vulnerable code across common tasks.
What the research found
Empirical studies evaluated Copilot suggestions against known vulnerability classes (e.g. SQL injection, cross-site scripting, path traversal, weak crypto). The results vary by language and prompt, but a consistent finding is: a meaningful fraction of suggestions are insecure if taken as-is.
| Study | Scope | Key finding |
|---|---|---|
| Pearce et al. (2022) | 89 security-sensitive scenarios in C, C++, Java, Python | Copilot produced insecure code in ~40% of cases |
| Asare et al. (2023) | Comparison to human developers on security tasks | Copilot did not introduce new vulnerabilities beyond those humans also wrote, but also did not reliably fix them |
| Sandoval et al. (2023) | User study in a realistic coding task | Participants with access to LLMs introduced similar or slightly more vulnerabilities than control group in that setup |
The takeaway: Copilot is not inherently secure or insecure, but it reflects insecure patterns from training data. Uncritical acceptance is the main risk.
Practical mitigations
- Treat suggestions as untrusted until reviewed (especially I/O, auth, crypto, SQL).
- Use secure-by-default snippets, linters, and SAST tools in CI.
- Add security-focused test cases for handling untrusted input.
- Prefer parameterised queries, input validation, and modern crypto libraries.
VERDICT
Copilot can suggest insecure code. Developers must review suggestions with security in mind and rely on automated checks.
Sources
- Pearce et al., “Asleep at the keyboard? Assessing the security of GitHub Copilot’s code contributions”, IEEE S&P 2022
- Asare et al., “Is GitHub’s Copilot as bad as humans at introducing vulnerabilities in code?”, Empirical Software Engineering 2023
- Sandoval et al., “Lost at C: A user study on the security implications of large language model code assistants”, USENIX Security 2023