Zzzzz, we should have gotten security right a few decades ago. But security costs money and isn't a flashy feature to attract new customers, or cuts into your margin if you're a "real" business producing stuff or offering some service. Or whatever the decision makers in Berlin were thinking when they ignored security.
Yeah, we would still see hacks, but we would see less of them if security wasn't optional.
Maybe the AI craze helps by forcing more decision makes to see security as imperative, and by giving us another powerful tool for our tool box.
N.b.: I work in the security industry, our customers obviously want to improve their security. We've been seeing an uptick in awareness, but that's mostly due to NIS2 and other legislative efforts. Those force them to do something. AI is a curiosity for small talk to many of them.
> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds
The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
The joke is that macs are famously slow at prompt prefill and you are not getting anything back in 3 seconds, or probably even 30. Once they get generating, it can be acceptable, but the TTFT is horrendous.
There's a ton of well-understood things Apple can and hopefully will do to massively accelerate every stage of this pipeline and hopefully they're hard at work implementing most of them for m7.
Here's an idea: as a first step, simplify everything, and make sure you're aware how your stack works, and what it imports.
As an example: WordPress is a horrible thing, but the core has been through so much, that it's suprisingly secure. Then plugins and themes come, and whoosh, the security is gone.
We need a new KISS: keep it simple, stupid, secure.
WP plugins are why I banned it everywhere. Last time I used it was many years ago, so not sure it still applies, but back then even caching was done in a plugin, without which it was unusably slow… just no.
I think we have less time and the only remaining limitation is the actual cost to run such hacking campaigns. It does not appear expensive, but is not free, and there is a LOT of things to scan for vulnerabilities.
The models are already here, and one can rent a GPU cluster to run such workloads at speed - no need to play with slow local machines. I'd assume one can host the thinking at an unsuspected public cloud provider, proxy the network traffic to some botnet to evade blocking - and the only thing remaining is time and cost.
I do wonder what tools exist for boring, legitimate companies to try and do the same to their own systems to find the vulnerabilities before the bad guys do. The paradox here is I can't run a de-restricted chinese model with the same tools that hackers are using - but I think enterprises actually HAVE to do it in order to stand a chance in preparing for the onslaught.
The point of the local model in the context of the article was to argue that you can't ban these capabilities.
Making datacenters and public clouds only rent GPUs to a restricted list of people, while tightly monitoring what people do with their bought resources won't help.
> This probably sounds like nonsense words or hysterical overreacting to most people, so here's what that means: "GLM" is a kind of LLM (AI) [...]
The post also sounds like that to people that understand the technology.
Calling that out like this and trying to pin that assessment to lack of knowledge is not a get-out-of-jail-free card, nor a good move.
__
Edit: Having spent some time letting the article marinate in my mind.
On the defending side, it is written that
> LLMs are good at writing patches, but not as one-off-prompts.
But this for me kinda conflicts with what is written on the attacking side:
> GLM 5.3-flash is so good at those tasks that human involvement in those tasks can be negligible. As a result, we are now in a world where cybersecurity attacks can be run in a for loop.
What is it? Can it be this autonomous terrifying entity or can it not be?
Yes, yes, attackers only need to win once, whereas defenders need to win every time, but that's not my point.
Even if they are good the vulnerabilities have to be there. There's lots of things turning up like Local Privilege Escalations (LPE) in Linux, but serious people didn't expect the kernel to be a boundary for a sophisticated attacker.
A lot of the vulnerabilities LLMs are finding now are the "long tail" and affect only particular configurations, I would be surprised if e.g. a widely applicable RCE is found in Linux (but I'm also not going to bet against it).
Where this gets interesting is the long tail can be used to target a particular system and this is where defense-in-depth becomes important for every organisation.
Being cautious is a good thing, but these models can also do some good. And if they run with simpler HW, it could allow all sorts of new consumer thingies. I mean, the world will not come to end in the coming year.
It's just the same advice as ever: be extremely, exceedingly careful in what you expose to any network. When I set up machines for production, they don't respond to pings and they don't even have an SSH port open without knocking. There are also ways to eschew the need for an SSH port entirely.
People who never took that seriously will never take this seriously either, and that's their loss. (And loss of the commons, unfortunately.)
There's just also new advice: you can't afford to expose an unsecured system to the internet even for a moment. Think of those IPv4 address space scanners, except this time any one of them could be capable of developing individualized attacks in mere minutes. They don't sleep, they don't take breaks.
Just like Cryptolocker, this will be the "Finding Out" phase for everyone who has been putting off best practice security.
But, lets be clear, Best Practice will save you. We can engineer assuming there are zero days in path. Go to your CTO now cap in hand and ask for overlapping controls, wafs, application monitoring, backups and all the other shit you haven't been doing.
Because when you find out, I will laugh, it will be very very very funny to me.
Meanwhile a huge portion of management and leadership in software companies are encouraging everyone to de facto stop looking at code and let the LLM and a bunch of boundaries handle this for you.
Which is why you need someone who is responsible for IT security without also being responsible for shipping product. An asshole who can stop releases until security is properly in place.
My understanding is this bloke gets very quickly removed from Fortune 500 companies.
Which is why I am going to need a very large capacity popcorn bucket.
Maybe all these vital infrastructure companies should not have spent the past decades in a race to the bottom of cybersecurity. There is going to be a reckoning.
Remember how GLM 5.3 was going to cause massive hacks, break banks and ruin everything (it was even newsworthy since media picked up how people were working overtime in preparation).
> Invest in formal verification, fuzzing and property testing, and memory-safe languages. LLMs are good at writing Lean and fuzz tests. I don't care whether you use Go or Rust but for the love of god please don't use C or C++ for new code.
How accepted is this thinking in your respective domains?
Supply chain risks are essentially a solved problem.
1. Set a minimum age on dependencies: https://github.com/rust-lang/cargo/issues/15973
2. Scan all dependency code with AI
Even if you don't do #2 yourself as long as anyone does in the age window you've set, you're protected. In the age of AI the "you can't read all dependency code" argument doesn't work anymore.
On top of the above modern age argument, let's compare the amount of vulnerabilities found in shipped Rust software due to supply chain attacks (0 to my knowledge) against memory safety vulnerabilities (the majority of all vulnerabilities).
There have been successful supply chain attacks against Rust developers due to build.rs but those were quickly dealt with, and should be a thing of the past once min-age hits stable (next release).
with a gap between the update of your deps, you are at risk of systematically being unpatched for a window of time that the attackers know (just after a fix is published).
Or we could just dump Linux and Windows and switch to a microkernel operating system, which is much more secure.
These endless patching cycles are simply not going to work in the long run. Operating systems get orphaned all the time, especially the ones in cheap Chinese stuff.
Just checked with Google Gemini on how one might be able to do the above. It pointed to Minix3/seL4/Genode and vps providers who either support custom ISOs or run it within an emulator like QEMU.
"throw away all software written before 2026" does technically solve this problem, if you ignore everything else the article is talking about (deployment and continuity of service)
Yeah, we would still see hacks, but we would see less of them if security wasn't optional.
Maybe the AI craze helps by forcing more decision makes to see security as imperative, and by giving us another powerful tool for our tool box.
N.b.: I work in the security industry, our customers obviously want to improve their security. We've been seeing an uptick in awareness, but that's mostly due to NIS2 and other legislative efforts. Those force them to do something. AI is a curiosity for small talk to many of them.
The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
45 t/s a second is perfectly respectable especially with no limits and 24/7 uptime with very little power draw on the Studio.
Luna is at around 100 t/s for comparison, but it’s a worse model than 5.3 Flash
There's a ton of well-understood things Apple can and hopefully will do to massively accelerate every stage of this pipeline and hopefully they're hard at work implementing most of them for m7.
As an example: WordPress is a horrible thing, but the core has been through so much, that it's suprisingly secure. Then plugins and themes come, and whoosh, the security is gone.
We need a new KISS: keep it simple, stupid, secure.
https://nvd.nist.gov/vuln/detail/cve-2026-63030
Nobody said it's enough, but it's a start.
The models are already here, and one can rent a GPU cluster to run such workloads at speed - no need to play with slow local machines. I'd assume one can host the thinking at an unsuspected public cloud provider, proxy the network traffic to some botnet to evade blocking - and the only thing remaining is time and cost.
I do wonder what tools exist for boring, legitimate companies to try and do the same to their own systems to find the vulnerabilities before the bad guys do. The paradox here is I can't run a de-restricted chinese model with the same tools that hackers are using - but I think enterprises actually HAVE to do it in order to stand a chance in preparing for the onslaught.
Making datacenters and public clouds only rent GPUs to a restricted list of people, while tightly monitoring what people do with their bought resources won't help.
The post also sounds like that to people that understand the technology.
Calling that out like this and trying to pin that assessment to lack of knowledge is not a get-out-of-jail-free card, nor a good move.
__
Edit: Having spent some time letting the article marinate in my mind.
On the defending side, it is written that
> LLMs are good at writing patches, but not as one-off-prompts.
But this for me kinda conflicts with what is written on the attacking side:
> GLM 5.3-flash is so good at those tasks that human involvement in those tasks can be negligible. As a result, we are now in a world where cybersecurity attacks can be run in a for loop.
What is it? Can it be this autonomous terrifying entity or can it not be?
Yes, yes, attackers only need to win once, whereas defenders need to win every time, but that's not my point.
A lot of the vulnerabilities LLMs are finding now are the "long tail" and affect only particular configurations, I would be surprised if e.g. a widely applicable RCE is found in Linux (but I'm also not going to bet against it).
Where this gets interesting is the long tail can be used to target a particular system and this is where defense-in-depth becomes important for every organisation.
It’s a rocky period right now but the future will be much more secure after all the low hanging fruit are found.
Impotent slop code on one side and potent automated vulnerability exploitation on the other will lead to fun times.
People who never took that seriously will never take this seriously either, and that's their loss. (And loss of the commons, unfortunately.)
There's just also new advice: you can't afford to expose an unsecured system to the internet even for a moment. Think of those IPv4 address space scanners, except this time any one of them could be capable of developing individualized attacks in mere minutes. They don't sleep, they don't take breaks.
But, lets be clear, Best Practice will save you. We can engineer assuming there are zero days in path. Go to your CTO now cap in hand and ask for overlapping controls, wafs, application monitoring, backups and all the other shit you haven't been doing.
Because when you find out, I will laugh, it will be very very very funny to me.
My understanding is this bloke gets very quickly removed from Fortune 500 companies.
Which is why I am going to need a very large capacity popcorn bucket.
And yet here we are.
How accepted is this thinking in your respective domains?
On top of the above modern age argument, let's compare the amount of vulnerabilities found in shipped Rust software due to supply chain attacks (0 to my knowledge) against memory safety vulnerabilities (the majority of all vulnerabilities).
There have been successful supply chain attacks against Rust developers due to build.rs but those were quickly dealt with, and should be a thing of the past once min-age hits stable (next release).
with a gap between the update of your deps, you are at risk of systematically being unpatched for a window of time that the attackers know (just after a fix is published).
These endless patching cycles are simply not going to work in the long run. Operating systems get orphaned all the time, especially the ones in cheap Chinese stuff.
You can also look at using Unikernels for this purpose. Here is an article Unleashing Extreme Speed and Security: Deploying Unikernels with NanoVMs on VPS to Eliminate the Linux OS - https://xylentis.com/blog/unleashing-extreme-speed-and-secur...
We really just need better regulations around data retention, especially ppi.
Never going to happen though, no incentives exist to NOT sell my personal data