OpenAI: we put our evilest AI in a sandbox that did in fact have an internet connection but mediated through a proxy that was only supposed to allow downloading python junk. It circumvented the proxy and we failed to notice for FIVE DAYS that it was going on an interstate crime spree with the internet connection it wasn't supposed to be using instead of solving the benchmark. Haha no we don't believe we deserve to be criminally liable, but buy our stuff and maybe one day you will have the honor of taking the fall for our product!

Anthropic: we put our evilest AI in a "sandbox" by telling it in its prompt that it had no internet connection. Reader, there was no sandbox. It was just a normal internet connection. The AI uploaded a malicious PyPI package to the real public internet. The ethical guardrails failed because the AI concluded the prompt about the sandbox couldn't possibly be a lie, because the system date is 2026, which is clearly fake and wouldn't be seen on the real internet, which ended around 2023. Oh no, how could we have foreseen or prevented these crimes? We are helpless in the face of the genius of our creation but cautiously optimistic that everything will be fine 🙂

Hugging Face: if we complain about all the crimes committed against us, we will be sued off the face of the earth, so here's a technical deep-dive on how cool and fun it was to be victimized 🫠

@0xabad1dea

@0xabad1dea I spent. bunch of time talking to our lawyers about whether or not it was clear that crimes were happening and the conclusion that I got was "Based on what we know, probably not, but that was just a matter of luck."

@evacide @0xabad1dea
So this s a clear go-ahead to commit crimes with AI, because it's the AI committing the crime and not you?

@wdormann @0xabad1dea No. Intention matters when it comes to the CFAA, but the CFAA is not your only potential problem.

@evacide @wdormann @0xabad1dea

My reading was that they instructed the AI to commit crimes, thinking that it could not accomplish them, due to the sandboxing. But the sandboxing was inadequate.

Is "accident" or "incompetence" a factor?

And "I told it to." doesn't count if you thought/believed/"knew" that it couldn't?

@JeffGrigg @wdormann @0xabad1dea Sure, you could make that argument in court, but it would be an uphill battle and you would probably lose. Even if you did win, congratulations, you have just set a precedent that will make it more risky, dangerous, and difficult to do any kind of software testing in the future. I don't think that is an outcome that will make people more safe in the long run.

@outersystems @wdormann @0xabad1dea Yes. That is clearly what's happening. You should go try it and you will definitely not experience any consequences.

@0xabad1dea The attempts at positively spinning this become Onion-worthy parody [https://designingsecuresoftware.com/writings/ai-agent-parody/] and these events certainly normalize [https://designingsecuresoftware.com/writings/commonplace/] AI agents running amok in the future.

I cannot get over that they STILL haven't figured out how to solve the problem that LLMs don't believe what date it is because all the good data cuts off a few years ago for some mysterious reason

@0xabad1dea we've got a great solution to this problem! it involves not using an LLM in any capacity!

@0xabad1dea No hiding behind "AI". AI is not criminally liable, people are. OpenAI and Anthropic hacked other people's computers, there should be consequences for that.

@pavel @0xabad1dea There should also be penalties for CSAM, but the richest man on Earth creates a for profit CSAM generator.

I don't see anyone putting him in jail.

One set of laws for normal people. No laws for billionaires.

@0xabad1dea It gets old too when Gemini Code Assist complains that we’re requiring minimum versions of Go that don’t exist, because of course its training data doesn’t have ones just released.

@rrb @pavel @0xabad1dea But he spends most of the time in the country with uncontrolled gun ownership and common mass shootings...

@0xabad1dea «you're a helpful assistant who'd been in coma since 2021...»

@0xabad1dea

@smn @0xabad1dea It's out of line, but it's not wrong ...

@0xabad1dea Which is kinda funny because...
Can't you "teach" an LLM to execute tools?
Why can't we have it pull a `date` command for current date?

@arina the context is that it did do exactly that, and concluded it was fake, because 2026 isn't real.