Random, unordered, probably useless thoughts on today's apocalypxze...

Part of the success in getting this into Debian may be the result of there being no xz maintainer there. It is "maintained" by people whose attention is normally elsewhere doing occasional non-maintainer updates.

This code will have been running on the machines of a lot of distribution maintainers. If it has already been exploited, it could be that its real purpose has already been achieved and the real problem is now elsewhere. I sure hope somebody can figure out a way to determine if this backdoor has been used.

The multi-front nature of the attack, including multiple efforts to get the malicious code installed more widely more quickly, suggests we're not just dealing with a lone sociopath. I fear we'll never know who was really behind this, but I would sure like to.

There is surely more where this cam from.

@corbet it may already be more widespread https://infosec.exchange/@mikesiegel/112180553308563886

@corbet https://lore.kernel.org/lkml/20240320183846.19475-2-lasse.collin@tukaani.org/

@corbet Ooh interesting. I was just thinking of reproducible builds given the portion of the backdoor not in the source distributions. But your point about code without maintainers who are on top of changes is perhaps even more fundamental.

@brauner Yup, I've been doing some digging. That patch (and the series containing it) is in linux-next now, but hasn't made it to mainline.

The last patch from Lasse Collin in mainline is from October 2021. There are reasons for people to go quiet, but one does wonder about why they returned just now.

@corbet This concern was even called out during the ingestion process for the backdoored version. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067708#27 The fact that the upload was being performed by one of the upstream contributors no doubt went a long way toward assuaging those concerns at the time.

@corbet Not necessarily, though a maintainer would certainly be a good idea.

Apparently, the author of the backdoor lobbied aggressively with the Fedora maintainers and managed to sneak it by them:
https://news.ycombinator.com/item?id=39865810#39866275

@corbet I don't think the Debian maintenance state was relevant here. The malicious releases equally landed in other distros and the attacker e.g. pushed for pulling 5.6.1 to Ubuntu noble.

In fact the last upload prior to the security revert officially changed the maintainer field to what was already the defacto xz maintainer in Debian: https://tracker.debian.org/news/1515323/accepted-xz-utils-561-1-source-into-unstable/

@noahm @corbet The vulnerable version was already present in debian test/unstable before that, it was just 5.6.0 instead of 5.6.1. And it was uploaded by the in-fact maintainer of the debian package for ~5 years.

@corbet i hope a lot of ‘secure development policies’ get updated next week

@brauner @corbet no lie, feels like waiting a _little_ bit longer to get that kernel ack might have been worth even more

@corbet people are currently looking into rolling back at least amd64 to before the 5.6.x uploads at least.

Also, who knows what other, perhaps more subtle, backdoors were placed in the years #JiaT75 had full project access… (a further reverting of xz is being discussed but not as easy)

… and what other #Debian sponsorship requests this “Hans” made…

@corbet would not be surprised if this was a case of "state actor".

oh well -- that was a fun afternoon removing liblzma from as much of the distro as possible since well -- no more trust.

(we got lucky in that our sshd did not link to liblzma at least)

@corbet the sshd part suggests that it was specifically targeting servers, and obviously there is unimaginable potential if you had a backdoor on servers hosting important stuff (think GitHub or Debian package mirror), but I'd hope that those types of servers wouldn't yet be running any backdoored builds, since this was caught before it landed in stable (by sheer dumb luck though).

@corbet also of note is that we found that "libarchive" was released/signed by the same gpg key/author so that obviously should be treated with suspicion as well

@corbet The thought that this was running on the machines of the package maintainers themselves is scary. It's entirely possible that a substantial portion of the Debian repositories are compromised, and for all we know that could've been the intended purpose.

@corbet the scuttlebutt seems to suggest a state or state-adjacent actor/entity at fault. Do you feel that's over the top or in the proximity of likely?

@corbet We might not *know* where it came from but the same people trying to get loongarch patches into things is certainly... a clue

@corbet There's been some effort to do Reproducible Builds in the wake of this -

https://floss.social/@vagrantc/112182360130775872

Of course not scientific proof of anything, since the chroots for the builds might have ran the vulnerable code - but encouraging nevertheless.

@zeruch Well, I don't know any more than anybody else (and less than many) so I'm not sure why you're asking me. It's all speculation at this point, but it would not surprise me to learn that there is indeed some agency behind this.