litchralee@sh.itjust.works 1 day ago
Answering the titular question, why would the advent of AI-found vulnerabilities substantially change one’s threat model? Before LLMs, the prospect of an undiscovered vulnerability was non-zero probability then, so what exactly has changed now?
In terms of attacker capabilities, it would be a mistake to solely observe that attackers now have LLMs and conclude that this means the threat landscape has changed. The use of LLMs does not magically create a root escalation where one does not or cannot exist. Part of your threat model should include defense-in-depth mitigations, so that when (not if) an attacker finds a 0-day exploit, they are severely limited in gained functionality on your machines. In the simplest cases, jails or containers serve to keep separate the web-facing processes. VM isolation would be a further step, because an attacked cannot gain machine control without both a software exploit as well as a VM escape route.
Phrased another way, the people who have done their threats analysis properly and have accepted the residual risk of hosting their own services that face the web, should not really be additionally perturbed by AI advances. This isn’t the same caliber of landscape change as would, say, the invention of a cryptographically relevant quantum computer. If the worst thing that can happen to your home IRC server is that it gets flooded by a script kiddie, AI does not mean your server can now burst into flames.
Much like with human hygiene, some basic rules are unusually effective against most threats. The equivalent to washing one’s hands is to stay on top of system and software updates. Institutionally funding the organizations (eg PHP Foundation, curl, USA CDC) so they can fix these problems would also help.
Before LLMs, the prospect of an undiscovered vulnerability was non-zero probability then, so what exactly has changed now?
The problem is not the prospect of an undiscovered vulnerability (there is always a vulnerability somewhere) but rather the prospect of a reduced response window.
There’s a wide gap of difference between “non-zero probability” (0.0076%) and “non-zero probability” (82.01734%).
If you’re dealing in probabilities, that’s not threat modeling; it might be in the larger umbrella of risk management though. Defining a threat model means to qualify what sort of attacker you expect to face and what they are assumed to be capable of. An assumption such as “this adversary can send arbitrary packets to the public IP address” is what would be documented in a threat model.
The raison d’etre of threat modeling is to methodically tackle the actual or assumed risks that an attacker can pose against you. It is not an exercise in swinging at shadows, though I wouldn’t discourage overstating the capabilities of an attacker, if needing to protect assets like patient info or PII. Doing threat modeling based on “gut feeling” is a recipe for not recognizing risk where it’s present and seeing risk when there isn’t any.
The fact remains: you cannot squeeze blood from a stone. Even the most sophisticated AI cannot steal PII from a computer that doesn’t store PII. Proper threat modeling means you don’t put anything on the line that you haven’t squared up against the risks, however improbable. For any non-zero concern, a good threat model will discuss its mitigation, whether it will be absorbed, or if it is an open question. For an out-of-scope concern, a good threat model will explicitly state that assumption.
Defining a threat model means to qualify what sort of attacker you expect to face and what they are assumed to be capable of
Which is exactly what OP is talking about. 5 years ago if you were only trying to defend against script kiddies, using containers was good enough. But now you have to consider that script kiddies might find container escapes.
.jpg){kind=link}
boudica@lemmy.world 13 hours ago
We’re talking self hosting, not an enterprise setup. Industry best practices and layers of defense mitigation are not realistic expectations for something like a media server that is only intended to be accessed by the owner and maybe a couple other people.
litchralee@sh.itjust.works 11 hours ago
I disagree: self hosting and enterprise are not mutually exclusive. In fact, “enterprise on-prem” is the marketing description for self-hosting. I do not accept any definition of self-hosting that gate-keeps to any degree.
It is incongruous to talk about “only home self-hosting” or “only enterprise on-prem” when it comes to security mitigations, threat modeling, and risk management: it is merely a question of scale.
Sure, the user base for a home setup might be single digits, but the attackers are identically situated: their capabilities do not grow or shrink based on whether they are targeting a bank or a Jellyfin server.