A lot of the current hype around LLMs revolves around one core idea, which I blame on Star Trek:

Wouldn't it be cool if we could use natural language to control things?

The problem is that this is, at the fundamental level, a terrible idea.

There's a reason that mathematics doesn't use English. There's a reason that every professional field comes with its own flavour of jargon. There's a reason that contracts are written in legalese, not plain natural language. Natural language is really bad at being unambiguous.

When I was a small child, I thought that a mature civilisation would evolve two languages. A language of poetry, that was rich in metaphor and delighted in ambiguity, and a language of science that required more detail and actively avoided ambiguity. The latter would have no homophones, no homonyms, unambiguous grammar, and so on.

Programming languages, including the ad-hoc programming languages that we refer to as 'user interfaces' are all attempts to build languages like the latter. They allow the user to unambiguously express intent so that it can be carried out. Natural languages are not designed and end up being examples of the former.

When I interact with a tool, I want it to do what I tell it. If I am willing to restrict my use of natural language to a clear and unambiguous subset, I have defined a language that is easy for deterministic parsers to understand with a fraction of the energy requirement of a language model. If I am not, then I am expressing myself ambiguously and no amount of processing can possibly remove the ambiguity that is intrinsic in the source, except a complete, fully synchronised, model of my own mind that knows what I meant (and not what some other person saying the same thing at the same time might have meant).

The hard part of programming is not writing things in some language's syntax, it's expressing the problem in a way that lacks ambiguity. LLMs don't help here, they pick an arbitrary, nondeterministic, option for the ambiguous cases. In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour.

There are use cases where getting it wrong is fine. Choosing a radio station or album to listen to while driving, for example. It is far better to sometimes listen to the wrong thing than to take your attention away from the road and interact with a richer UI for ten seconds. In situations where your hands are unavailable (for example, controlling non-critical equipment while performing surgery, or cooking), a natural-language interface is better than no interface. It's rarely, if ever, the best.

@david_chisnall I think something that laypeople assume about LLMs is that they are smart enough to figure out what they mean from ambiguous language. People that don’t work with programming or other computer tasks assume it is easy for the computer to “just make it understand what I want it to do” when in reality it is almost impossible.

Time to share this lovely @CommitStrip again!

commitstrip.com/en/2016/08/25/…

@david_chisnall

@david_chisnall We really should give language interfaces a chance. Programming languages and existing interfaces are cool if you can type fast and have nice big screen nearby.

Driving car is one example, walking / riding is another. I'd really like to do basic stuff (play music, give me weather, take a note, summarize latest news about recent plane crash) without screen/keyboard and current technology should be good enough for that.

Heck, I'd like a cellphone that's 60 grams, not 200 grams and actually fits into pocket. That likely means small screen, but that's a good tradeoff for me.

(Plus LLMs have other uses. They are quite good at translations, for example. Of course, people are trying to use them for unsuitable stuff, too).