ngl it never quite hit home just how directly shareholders drive the direction of organisations. Seeing all the new windows laptops qcom and Microsoft are announcing, the "copilot+" bullshit, it's just so disconnected from what consumers actually want.

but hey, new things, big changes, wooo hyyype... The underlying message is that Qualcomm have just announced a brand new avenue for growth in the somewhat stagnating silicon industry. Now we can all go build "NPU"s and tell consumers the hardware they have is old and slow and shite.

Meanwhile M$, QC, and even intel/AMD get to leverage the aesthetic of growth, and line go up.

@cas I'm a staunch capitalist but to me this stuff reeks of corporatism rather than capitalism.

An absolute basic tenet of a free market is that participants are adequately informed. And when lies and bullshit dominate plus a vested interest to suspend disbelief in said lies then things are breaking.

That's basically a form of corruption, which does seem to be rife these days, LLMs only being the latest iteration - magic trick that appears to work is more than enough to keep all this shit going, for now.

Then onto the next... market can be irrational longer than sensible market forces shorting companies doing stupid things, as the absolutely correct but nonetheless doomed tesla shorters can tell you over many years.

@cas 1. yes
2. I'm still optimistic that there will be at least a few good things coming out of this. Like - I do want my hardware to be able to run stuff locally and not be dependent on the cloud. So having a NPU that makes certain common things faster/more efficient *is* something I'm looking forward to.

@cas I love how bigtech is completely disconnected from reality. "AI" stuff that people actually want is maybe an OCR or facial recognition for their home security.

All other stuff like LLMs that companies come up with is completely irrelevant to what people want, and consumers are so sick of it that they avoid products advertising "AI" altogether lol.

Take Windows for instance, even the most devoted Windows users among my friend group decided "enough of this enshittification, I'm giving Linux a shot" - which is of course a big win for us, as more people using it mean that maybe big companies like Adobe will start releasing releasing their software on Linux (not that Adobe is good, but you know - if "industry standard" software will run on it, it will naturally drive the adoption).

Long story short, investors and their demands introduce enshittification in companies, which is basically slowly killing them. Since we have good momentum right now, it is entirely possible that opensource will become a standard for desktops in fairly near future. After all, even governments in Netherlands, Germany and... China are pushing for it's adoption.

What this basically means is that thanks to enshittification we will have more open and transparent technology, and I'm very happy to play my part in it

@cas I believe what (most) consumers "want" is heavily influenced by what's available. In other words: what's advertised as available. People will buy these simply because they are going to be sold everywhere.

@ljs we can definitely agree that companies (tech especially) should not be allowed to get away with a lot of the misleading crap they do.

The free market ideal to me seems impossible as long as we allow for profit growth to be the driving incentive for companies; especially given the level of worker exploitation (especially in the global south) to train AIs, and of course the broader issue of the growing wealth gap.

Really what I'm saying is that if we want what's best for the 99% of workers and consumers, they should be the ones making the decisions (whether that be through worker owned co-ops or other methods of reducing and abolishing hierarchies).

@cas i believe it's more than likely also intended to be a surveillance chip which is the other more insidious reason US tech monopolies are very worrisome

@cas tons and tons of discussion about how it's intended for always-on use cases with external sensors with a distinct power rail in the white paper

@cas not just me, there are analyses beyond the purely technical too https://circumstances.run/@hipsterelectron/112482975521122360

@hipsterelectron recall is bad or a lot of very obvious reasons, no doubt. But

> I believe it's more than likely also intended to be a surveillance chip

This is just conspiracy shit. We're pretty familiar with how govt surveillance works nowadays, and smartphones have had always on features for a decade already. Literally the Snapdragon 845 had the first "sensor low power island", a part of the sensor DSP which could run all the time to collect reading and implement all the neat features like waking up your phone when you pick it up, and figuring out when it can get away with suspending for longer periods of time.

The 2022 version of this included integrating the entire camera stack with special hardware features to allow for the same sensor DSP to do face detection and lock your phone when you aren't looking at it.

I feel pretty strongly that this is just yet another instance of growth being prioritised over things like privacy and security. It feels pretty reductive to suggest that any of this stuff is motivated by it's potential use for surveillance, that would be some insane puppeteering to pull off.

@cas Hasn't that always been the way it goes in tech though? Customers would just ask for incremental improvements like faster single core performance, but it's hard to keep delivering such mundane things and making a profit.
So, you have to build something they don't want yet, market it aggressively including making it the only option, and hope a compelling use-case comes along soon. If it does, more people buy the new tech, if not, you try the next thing.

@crispybrown yeah nothing new, except my growing existential dread

@cas i provided a citation describing it as an outgrowth of the DRM framework of vista. "we're pretty familiar with how govt surveillance works nowadays" yes it's often done by partnering with US tech corporations the way RSA was used as a mule for DUAL_EC_DRBG or how google/fb/twitter and especially ms are regularly required to offer up user data with a gag order to stop them from telling anyone about it.

i won't be mentioning this in your replies again but dismissing it outright when microsoft is essentially synonymous with the US war machine is absurd. my mistake was attempting to appeal to technicalities because i'm not able to debate you in a learned manner about that. everyone on the fediverse has been mentioning the ease of abusing this technology for surveillance in the context of stalkerware but it's somehow going over the line to say its potential for abuse was a design goal? the way DRM content was explicitly called out as being safe from recall is a very strong argument to view it with the same concern as hardware for DRM enforcement.

@hipsterelectron I don't disagree with your point. I'm just really wary of this whole topic because

a) I'm not hugely well researched on it (and i don't trust my memory)
b) it goes from realistic and reasonable concern to unhealthy conspiracy bs so quickly, and I don't know of trustworthy sources.

will three letter agencies use this tech for surveillance? almost certainly if they can. But was this tech pushed by them specifically? much harder question to answer imo.

@cas "we're pretty familiar with how govt surveillance works nowadays" yes it's often paired with "AI" which is why meredith whittaker works at signal now after seeing google legal focus on "AI" to reduce the liability of producing a surveillance machine. calling it "conspiracy shit" is a fucking wild thing to say especially without even considering the DRM analogy i linked to

@hipsterelectron thinking more on why i feel so strongly about this point in particular, i think I'm really demotivated by it. There is nothing I can do to counter this at a systemic level that I'm not already pushing for with FOSS.

@cas Also "NPUs" have been a thing for a couple of years in smartphones already afaik. I think they were mainly used for speech/face recognition, I guess also camera/image postprocessing and similar relatively "useful" "#AI" features.
Interestingly only now with #LLM s Microsoft makes this a big marketing thing.
Probably they are just behind #Google/#Android, and now that they get the hardware for it too, they are focusing on it like crazy...

@lw64 @cas not on the same chip as the CPU and not after the US just spent many billions of dollars on the CHIPS act to move TSMC operations to the US ostensibly to stop "china" from building chips to perform surveillance of US citizens

@hipsterelectron @lw64 ??? the CHIPS act from what I've heard is mainly about making sure we aren't all screwed the next time there's a big tsunami in Taiwan lol

wdym "not on the same chip as the CPU"? All the DSP stuff I talked about is on one die in the case of Qualcomm SoCs.

@lw64 @cas note however that i recant my ridiculous initial reply https://circumstances.run/@hipsterelectron/112504996570463288

@cas @lw64 well i will absolutely be noting everything you've told me since i have been on wrong on literally every single technical detail

@cas @lw64 i misinterpreted some marketing drivel from the white paper which if anything should show me that your interpretation of this being bog standard corporate hijinks is more likely to be correct

@hipsterelectron which white paper? out of interest

@cas https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Unlocking-on-device-generative-AI-with-an-NPU-and-heterogeneous-computing.pdf from this marketing page https://www.qualcomm.com/news/onq/2024/02/what-is-an-npu-and-why-is-it-key-to-unlocking-on-device-generative-ai. what i drew from it was that technical especially hardware monopoly serves both the purposes of capital as well as hiding other stuff being done in the background (i was thinking of their "always-on AI" mention from this paper). i absolutely see why you would react the way you did because yes obviously any modern device correctly has always-on functionality and i'm positive you've had to deal with annoying losers who refuse to contribute but will bother you about these sort of necessary features being enabled by default. i thought "always-on AI" seemed (in the vein of DRM) to be slightly less obviously useful but i too see how i was misinterpreting their marketing "AI" drivel as being meaningful instead of meaningless

@hipsterelectron urgh yeah this is just all the (actually very cool) tech they already had just marketed for AI. Not super interesting (i mean the tech is but this angle is meh).

and yes, i have had people try and tell me that Qualcomm SoCs have backdoors or are insecure because the modem is integrated into the SoC. There's totally some interesting conversations to have there, but I'm not a security expert and the folks in question there were clearly just spreading FUD.

it's something I'm wary of, particularly with what you choose to focus on. Concrete examples come across much more earnest and sincere than vague concepts (and make it much easier to have a productive conversation)

also lol at this image in the paper, the chip in question is the Snapdragon 410 not the 820 (that was codename msm8960), and they are literally gaslighting us here. This SoC was the first to include a Hexagon DSP (their cool custom architecture which all their modern co-processors including the modem, NPU, video transcoder, GPU, camera stack, etc are built on). You could offload some basic tasks to it but it is blatantly not an "AI engine", it was the audio DSP and just had an interface for running custom code.

@cas you have been incredibly patient with me and i really appreciate it and won't make you regret it

@cas
As someone who frequently needs to evaluate Gibbs Sampling models the NPU accelerators are super disappointing, both from a model building and a model execution standpoint.

The RISC changeover wasn't happening on its own, because desktop environments don't really value power consumption. ARM clearly thinks that consumers needed a little something to sweeten the deal, and apple has x256 acceleration chips tightly licensed.
Normal folks aren't buying desktop/laptops at all anymore though...

@dnavinci that's a shame to hear, but not super surprising i guess. Do you know if these mostly software/interfacing issues or is it an underlying problem of the architecture?

@ljs @cas there is an amazing anecdote in "Factfulness" which boils down to "if you want companies to behave ethically, you need your gramps to tell their pension funds to elect ethical board members who will appoint ethical CEOs". It definitely jived with my impression (as an European) that the US will never be able to escape the worst aspects of capitalism as long as retirement income will be tied to the stock market -_-

@cas @hipsterelectron To be fair, ability to power down modem chips (Librem) and the need to explicitely pass audio samples to it (aka N900) makes security analysis easier and is a good thing.

@jpetazzo @cas regardless of your views on capitalism or the stock market, it is in nobody's interest - a retirement fund's or anybody else's - to be putting money into emperor's new clothes bullshit.

@cas remarkable that on fedi those of us with disparate economic/political beliefs can have civil discourse and find common ground :) that's not quite as prevalent on twitter...

@cas
The architecture of the NPUs are much like the Coral USB sticks. They can only -execute- models and only contain a small fraction of the tensorflow instruction set.

At least for me, a bio-sensing regression developer, I end up with a pretty short iteration cycle in early R&D.
If my model works then it's generally up to an engineering team to package it up for consumers, and they tend to keep my models on our servers (with GPUs), rather than on client devices with mini acceleration cards

@dnavinci huh i see. the older compute DSP Qualcomm had would let you run arbitrary code, i assumed the NPU would be the same.

@cas
Yeah, they frame it as "arbitrary" but when you get right down to it, it only supports TF up to version like... 3.1 or something like that. And the line between 3.1 and real current version is "tensorflow-lite"
Some folks discussing here:
https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868

@dnavinci i think we are taking about different things, as the traditional fastrpc mechanism does literally let you compile C or C++ code for the DSP and run it (via RPC). I guess they locked it down on the newer stuff, maybe just at the OS level though