Sorry Paul, I should have define a LLM. First LLM stands for Large Language Model.
To me a LLM is a vast array like a 2 or 3 dimension table like we knew in the programs we use to write. But really, the LLM is so much more vast and complex. And the LLM is nothing more then numbers. Tokens are converted to a number and the tokens have a weighted value associated to them. This is a large over simplification and may not even be 100& correct.
Here i asked AI to take my wording and reword it to more accurate, but still at a high level of undersatanding.
Please don’t be for me. It’s a long time since I did any programming and now my research is only very simple stuff.
We organise events in our village and our former admin member did the posters in PowerPoint and converted them to jpg for me and the site. Our new replacement uses AI to generate the images but takes him much longer and appears more completed especially if details change such as hour or DJ to a group. Not been asked to do it myself but guess I would use libreoffice and its version of PowerPoint. Not AI
Well, the model (LLM) can and does make mistakes. But you have to remember the LLM has been trained on millions upon millions of sentences. The LLM can predict the next token for a word, part of a word, or part of a sentence with huge accuracy from the calculations the the highest possibility from all that training..
I type to an AI and ask it to complete the sentence.
“A cat is a”
So, after all the millions of examples the LLM was trained and the number of times it saw (feed tokens for) “A cat is” what was the most often next word and it was “pet”.
“A cat is a pet”
The next highest 4 tokens after cat was; cute, playful, furry, small.
A cat is cute.
A cat is playful.
A cat is furry.
A cat is small.
So that is how the model avoids mistakes or nonsense. It picks the next token (word) that appears most often.
Why didn’t the model come back and said 'A cat is round."
Because the probability is near zero. Not impossible but highly unlikely.
Answer form my understanding and a chat with AI. Example of 'A cat is" supplied by AI.
Often the tokens are subwords (‘tion’, ‘mis’, etc), so stringing together two subwords for example is much less likely to result in a misspelling than stringing together eight individual letters one by one. Like it’s easier to build a Lego train if you’ve been given two big pre-assembled pieces than having to build it bit-by-bit yourself. Plus the vast majority of the training data will be spelt correctly in the first place and models tend towards statistical regularity. If the correct spelling corresponds to higher-probability continuations, a decoder will usually pick it. If you increase temperature or allow sampling to be more exploratory, the decoder has a wider range of acceptable solutions to the next token, so you might see more spelling ‘mistakes’ per se.
Interestingly, I find that when I read AI generated fiction (typically not by choice!), you find that there are zero spelling errors, but you do get a lot of meaningless sentences… Words that are each correct in isolation and probably cluster close together in the training corpus, but when strung together are simply nonsense.
So in a large slab of text, like an ai book, there will be rare grammar errors or more correctly bad sequencing of tokens.
My guess is it is good at predicting the next token, not so good at predicting the next plus one, and so on.
or
does only ever predict ‘next’ then add that and loop around?
A very good question. I can see you have your thinking cap on. It also got me to thinking about how an AI responses to a question or input. I have been using 3 different AIs for different reason, but one was to see how the AI reacted. The 3 AIs I been using are Copilot, Gemini, and ChatGPT.
Over the past 3 months, I ave been using Copilot the most and have been setting rules and memory in it to response to me in the way I prefer. So taking you idea, I thought I would pose the simple “The cat is a” to all 3 AIs to see what the difference might be.
ChatGPT
Gemini response to The cat is a
Copilot
Different AI can give different responses. In this test the input an ambiguous text so different replies were expected. AIs response to a direct question or problem usually match pretty closely. That has been my experience so far.
We have talked a lot about LLM or model inside a modern AI. Here are the layers of a AI according to an AI. The LLM consists of items 2 thru 6 working as one unit.
And of course if you really want a good explanation of how AI doesn’t work, or other fun informative non-information, you can always consult Halupedia.com It is a genuine artificial idiot designed to produce hallucinations…
I use AI mostly for silly things. Image creation and such. However, I do use it for things like summarizing long emails and articles. Asking technical questions. I sometimes test the various AIs by asking questions I know the answer to in order to see if the result matches the actual answer. I only pay for Perplexity Pro. I may do ChatGPT as well. Not sure yet.
I found a decent app that allows me to have all my AIs open at once or filter through and use the one I want, AI Gate.
This short quote may well apply to AI , as well as to computers
“Computers are useless. They can only give you answers.” –Pablo Picasso
There is indeed some shortsightedness in llm applications. … it is great at answering verbal questions, but lousy at maths or graphics or anything requiring an element of insight.
Yes Neville you are right, AI can be bad at some of the basic math problems. Why? Because AI does mot compute the answer like a computer should do, instead AI predicts the answer base on what the most probable answer is.
None of these are things I would know about.
On the other hand, some of AI mathematical success are (source Gemini);
o 2024 Silver Medal Performance at the International Mathematical Olympiad (IMO)
o 2025 Gold Medal Breakthrough at the IMO.
o Perfect Score on the Putnam Exam (2025)
o Automated Formal Verification with Lean.
o Solving Long-Standing Open Problems
Some of the items from Copilot;
Erdős unit‑distance disproof
Cycle Double Cover Conjecture proof
Navier–Stokes claimed solution
Fermat’s Last Theorem full Lean formalization
Counterexample to the Jacobian Conjecture (dimension 3)
Counterexample to the Dinitz–Garg–Goemans conjecture
There are claims it stole the main steps to a solution from an Australian mathematician
Another issue with AI is that it has no concept of space or time. All it knows is the text it was trained with on a particular day.
We do the observing and we make abstract concepts like distance and time interval and velocity, and we write about them . Then AI reads our physics textbook and is able to navigate a drone or to teach someone physics … but if we ask it what space is it repeats what we have written???
But yes, it has impressive performances in some areas.
That is what a lot of people do not realize. AI is only a massive storage of weighted values from information it was trained on. AI does not compute, AI does not perform any type of reasoning.
Yes this was my error and not AI’s. I was listing some of the highlights with out reading all the detail. Below is what AI reported.
Navier–Stokes claimed solution — OpenAI reported an AI‑generated finite‑time
blowup construction for the Navier–Stokes Millennium Problem, formalized in
Lean. Status: disputed but historically significant.
So AI claimed a solution, but it has not been accepted.
Nearly all of human knowledge is stored as text ( a small amount as numbers). We dont keep much in our memory.
When we answer questions, we look up text , then we add reasoning and judgement.
When AI answers questions , it looks up text too, but in a different way to us, and it has nothing to add. All of the seeming commonsense in its answers comes from its method of extrapolating text.
The weighting it gives to various parts of a text sequence is the clever bit … they call it ‘attention’ . I dont like that word … it is a numerical weighting. I dont yet understsnd how they derive the weights.
Much as I f–king hate AI - I did a google search this morning - my Ubuntu 24.04 desktop will randomly switch my wallpaper to a black background - pure black… Nothing I can do to fix it or change the wallpaper - if I change the wallpaper - it’s still blank black… Happened at least half a dozen times…
Tried a few dconf commands - didn’t fix it… Previously I’d reboot…
Google’s AI suggested :
Press Alt+F2
Press “R”
and it will restart the gnome desktop… and it did and it left all my stuff running too!
It also mentioned this doesn’t work on Wayland - on Wayland you need to log out and log in again (might as well reboot I reckon) - yet another thing in favour or xorg/x11…
I reckon I’m gonna stick with xorg for the forsee-able future - if Ubuntu make that untenable - then I’ll have to switch distros…
