Days since an "AI found security bug" turned out to be totally false due to the inability of the tool to actually parse C code: 0

I'm seeing multiple of these type of "reports" per week now for Linux. Why do people think that an LLM can somehow do better than a compiler and also not even test their proposed changes to verify they even do anything?

{sigh}

@gregkh Some people simply lack the skill, but they'd like to add “contribution to the Linux kernel” to their CVs.
Disclaimer: I've no idea if that was actually the case here.

@gregkh I think it goes like this: Fame and fortune awaits whoever actually reports a security problem in the Linux kernel. There is no cost to the reporter in making an attempt, even if the attempt doesn't succeed. So using an LLM to generate what looks superficially like a good report means they have a chance of benefiting, and there's no downside in trying.

Same problem as with email spam, that is.

@gregkh ban early - ban often.

@gregkh so you're implying that those people actually "think" before submitting such reports...

that is very generous of you

@liw @gregkh regular spam is easier to catch and filter off though... This is a growing kind of attack on Open Source projects which drains resources and is hard to combat effectively.

@liw @gregkh there is more. You are coming from a different worldview than the reporter. The reporter, in a lot of cases, *genuinely believe that these tools are super powerful*. They are the AI of your movies. It is a belief I have seen everywhere in my circles of friends. If the AI "discovers a bug" then it has to be real and exist.

Validating does not even come to their mind, because "who am I to doubt the powerful machine". In their mind, they are the inferior, and just the messenger.

They cannot even *imagine* it could be wrong that much, or that validating it is possible.

@liw We assign 13 CVEs for Linux every single day. "Fame and fortune" is not something that happens for any of those reports, as a CVE is trivial to get if you actually want to just fix a kernel bug for real.

@ptesarik That's what `drivers/staging/` is for, we just took 10+ patches for that subsystem from new submitters yesterday. That's much easier to accomplish than trying to parse the output of an "AI tool" :)

@gregkh @liw doesn't need to be true; the kind of people that would do this kind of thing believe that's how it works, which is all that matters
if they had the braincells to realize it's not the case, they wouldn't be thinking of doing this to begin with

@gregkh And don't understand why these people are submitting garbage AI report.

What's the goal of it?

@gregkh this seems to be a very active topic right now

@gregkh you know the adage that as soon as a measure becomes a target it stops being a useful measure? I think something like that has happened with bugs and bounties

@gregkh I kind of doubt that they are capable of even testing it, or else they wouldn't use the lying machine in the first place.

@gregkh Yeah, I was trying to be funny in a sarcastic manner, again, and failed, again.

@gregkh

Full ACK.

Sad but true. 🤢

@gregkh Some people easily fall for marketing pitches.

@gregkh
We've spent billions of dollars on AI! You MUST use it, and believe its every pronouncement!

@gregkh I wonder if LLMs are going to cause more problems under authoritarian regimes, where people are conditioned to do what they're told without question. Seems like perfect conditions for modern "AI" to cause all sorts of havoc, with all of it being excusable with "the computer told me to".

@gregkh Yes, let's promote staging (again)! Sounds like a good plan to me.

@liw @gregkh I’d think the downside would be the reputational risk of being known to the maintainer(s) as the jerk who didn’t verify a vulnerability before reporting it.

If I ever thought I’d discovered a kernel vulnerability I’d be checking it over every way I knew how before submitting a report.

@gregkh Maybe "Days" should be changed to "Hours"?

@tisha @gregkh Bug bounties usually (and I've seen a report where it showed that some large companies pay out often enough even though the report is bogus).

@gregkh Perhaps someone should tell that to Sasha Levin, as he applies bad patches to AUTOSEL based on LLM output? :-(.

@Di4na @liw @gregkh What I don't understand in that context is why they believe the role of messenger is at all valuable in that case. Surely if the tool is so powerful and easy to use, the maintainers would already be using it themselves?

@gregkh
To test it requires actual work?

@gregkh fwiw, curl just bans these types of people https://hackerone.com/reports/3230082

@gregkh I suspect you've already seen this slide from @badger but just in case, or for anyone else reading this who doesn't (yet) follow him...

https://mastodon.social/@bagder/114856434115222517

@ardaxi @Di4na @liw @gregkh this is definitely not logical, but also most people are not logical unfortunately. Same sort of thinking as someone replying to a question with "I asked ChatGPT and here's what it said".

@mikaeleiman @gregkh ...and/or some companies?

@gregkh linters literally do their job better than a speculation machine

@winload_exe @gregkh Almost, but not quite, as if linters and other tools were carefully designed to do a particular job, and thus do it well.

@gregkh This is the grotty side of #LLMs, of course. There is a good side. Sometimes.

But mostly what I see is #AIslop, and because of that I give It about as much respect as I did the #BoredApe bubble.

If LLMs are to be taken seriously, their act needs clearing up!

@gregkh it’s the very beginning of a ddos attack

@Di4na @liw @gregkh it's (hard?) work to verify/validate the bug and it's needs skill ...

@ardaxi @liw @gregkh "surely, if my religion is the right one, everyone would convert on their own". Basically, they think we are little kids that have not understood the Truth yet. It is their job to lift us from our limited ways.

@gregkh Fun story... One month, it was my job to run Klokwork (static code analysis) against our own code, because somebody in management had decided it's important to fix all "vulnerabilities" that an automated tool can find. An expensive tool, mind you.
Two senior engineers and lots of build resources, for a month, and we changed hundreds of thousands of lines of code (some by script).
1/x

@gregkh After all that, the Product Manager did not want to merge it into production/main, because "too many lines of code changes".
I learned a lesson - when tasked, always ask "if I do this, will you ship it" of your Product Manager. Or just take the money to waste time...
But for fun, I ran Klokwork against the linux kernel source (we were cross-compiling an ARM kernel and rootfs/dist of our own) and the "violations" were voluminous.
But somehow nobody was worried about that.
2/x

@gregkh I don't think of LLM-based coding "assistants" any differently - in the hands of experts, probably useful. In the hands of ignorant, lazy people seeking quick solutions, dangerously untrustworthy results nobody wants.
And a distraction from efforts that could really improve your software or service.
3/3