New take on AI safety: prompt injection is social engineering
Understand this story
An article argues that prompt injection is a new type of security weakness introduced by large language models, and that in essence it is a social engineering problem: attackers manipulate models using the old tricks of deceiving humans, such as exploiting the model's empathy, disguising the source of information, and gradually building trust. The view holds that addressing this type of risk requires shifting from prompt injection defense to a "capability safety model" approach. This claim comes from the article's author and has not been independently verified.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI Security Issues Are Social Engineering Issues
The author argues that prompt injection is essentially social engineering applied to silicon, since LLMs, like humans, can be tricked by smuggled instructions that exploit modeled empathy or trust. He contends that capability-based security, which gives each program only the authority it needs, is a necessary but insufficient fix, and notes that projects like CHERI, Fuchsia, Capsicum, Cap'n Proto and Spritely have laid the groundwork. He hopes AI's ability to re-engineer software will help move networks onto this firmer foundation, while stressing that human flaws must also be addressed.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 12 23:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.