This article explains how ram supply issues are affecting computer prices
It also gives some insight into why AI needs so much ram and so many AI centres.
Not just computers effected,amazon kindle reader as well guess the next will be electric cars and guidance systems
I deleted the duplicare topic ā¦ no idea how that happened. Thanks
I really hope this AI bullshit is a bubble - and it bursts - leaving āinvestorsā empty handedā¦
I LOATHE the whole idea of it allā¦
All AI is really is gigantic algorithms leveraging against Large (almost wrote ālardā) Language Models - itās not AI - itās still ages away from being self aware or the āsingularityāā¦
Hence why these qāts want to deploy DC after DC - to run their LLMsā¦
I hope the bubble bursts - it seems worse than the shonky ādot.comā bubble from 25ish years agoā¦ They have nothing, they offer nothingā¦ and all of that vast compute power being harnassed to help some gronk find a shop, or get assisted to find āthe right shopā - when it could be use for āNON-evilā stuff like genetic analysis or huge scientific questionsā¦ But no - the ātech-broāsā are going enslave it to sell you more shit you didnāt wantā¦ Rah rah rah!
My employer is pushing pushing pushing all of us wage slaves - to try and get bullshit AI certifications and use it more and more - NO! I donāt want to do that! I have VAST ETHICAL concerns about AI - but the corporate twonks donāt - unless itās the HR twonks appearing āpolitically correctā by asserting theyāre following diverity guidelines and we should all feel safe because they respect my choice to be non-binary or self-select my preferred third person description - FāK OFF - you corporate FāKs donāt give an arse about any of that - but you can still spew out the hideous mangling of language trying to show you careā¦
BTW - I respect anyoneās choice to be referred to as āthey/themāā¦ I donāt careā¦
English would easier if every third person reference was āthey/themāā¦ āhe/sheā and āher/himā are the last vestiges of French into English, which is a non-gendered languageā¦
I never got my head around conjugation of French verbs and nouns - as all of them were gendered - e.g. a āboatā (bateau) or a cake (gateau) was a āblokeā, but a river or an ocean was a āchickāā¦ Yea Nah! the āeauā rule was one of the few I remembered - i.e. a ābureauā was a bloke 'cause it ended in āeauāā¦
Maybe Mary Antoinette didnāt say ālet them eat cakeā (a cake is āgateauā) maybe she was talking in Italian and said ālet them eat gattoā (pussy)ā¦ ***
*** BTW - I know the alleged quote of Mary Antoinette saying ālet them eat cakeā is apocryphal - she never said that (as far as we know) - something I learned watching āQIā on the ABC (Australian Broadcasting Corporation/Commission) from the BBCā¦
Try getting acces to some DC for a real scientific compute job ā¦ I bet you would have to disguise it as AI.
What OS do these DCās use?
Are they one computer or many in parallel?
Now much memory can one job access.?
How come those huge matrix multiplies they do dont run out of presision?
Why not build one really. big DC instead of 500 scattered all over the world.?
I expect that changes in the algorithm or in hardware will make them all redundant in less than 10 years.
I remember when crypto mining really drove up the cost of GPUs and absolutely humped the second hand marketā¦ But AI has driven everything up so much, itās wild. Drives that cost less than a hundred quid a year or two ago are now going for three, four hundred pounds. Mental.
Youāre already seeing it in the console gaming market, unfortunately. Steam Machine, xbox, and so forthā¦
Feeling very similar to you as well. I recently attended this interesting talk at a local book festival, which kind of went into the investment side of it and itās totally mental to me.
Thereās this circular economy propelled by Nvidia ā they help guarantee the financing that lets their own customers borrow pure massive sums to buy more GPUs, and then those customers invest more in AI, subsequently require even more GPUs, then borrow even more money to buy them from Nvidia, and Nvidia helps guarantee that financing againā¦
It kinda reminds me of the subprime mortgage crisis. And at least with subprime, there was an underlying housing market that already like, existed? Here, companies like Anthropic are committing tens of billions to future compute on the assumption that an end market of corresponding scale and profitability will materialiseā¦? Based onā¦ what, exactly?
Interesting article. I learn something new call HBM (High Bandwidth Memory) RAM.
Instead of of a 64 bit channel of normal RAM, the HBM uses 1024 to 2048 bit channels.
Which of course, makes them much faster.
Because it is like the ARMs race. Each major country wants their own super power AI center. Also business are paying big bucks for access to tailor AI centers. So the companies who build and supplies the companies with this service are making huge profits.
So it boils down to supply and demand.
I am not overly fond of AI, but I do believe it is here to stay. We may not like the idea of what AI is doing sometimes, but back in the 50ās and 60ās there were probably a lot of people who did not like to see the mainframe computes being installed in business.
Just imagine the number of people the mainframe compute replaced in Accounting, Payroll, Inventory, Procurement, Billing, and Customer Service.
Some of AIās usefulness (credit Copilot).
- Protein structure prediction ā AlphaFold
- De novo protein design ā creating new proteins
- Largeāscale language modeling ā LLMs
- AIāaccelerated drug discovery
- AIādriven materials discovery
- Autonomous scientific reasoning
- Generative design
- Climate modeling
- Fusion research
- Astronomy
- Mathematics
- Economics
Also 2 Nobel Prizes directly linked to AI;
2024 Nobel Prize in Physics
2024 Nobel Prize in Chemistry
- Well, call centers are being replaced by AI.
- Also ā provided it is used correctly ā LLMs can really help software developers.
- LLMs can also help analyzing a huge stack of forms with a bunch of open questions.
- It can also help to analyze and/or summarize a text. Yes, this is open to abuse, like the introduction of any other tools. Weāre learning to catch such cases.
- It can also be HELPFUL to teachers at various levels of schooling to (ironically) check for AI usage (cheating), but also help grade the papers and the tests the pupils/students turn in.
Problem is, a lot of users are abusing it. Cheating at school, for example, or using AI to formulate a legal argument by lawyers to name some.
But, yes, it really is a bubble. The AI companies are investing waaaaaaay more money than they have, and they canāt even generate enough profit to cover the interest of the loans theyāve acquired. Once the market wakes up to this fact, there will be a huge reckoning and a lot of companies will go belly-up and even more people will lose a huge amount of money.
Companies like Google, Amazon, and Microsoft will probably survive the AI-bubble, but OpenAI will most definitely not. OpenAIās sole product is AI, and when the reckoning arrives, it will all of a sudden have to repay a lot of debt or show that it can pay the interest on its debt. It will not be able to do so, and hence itāll fail. There are several others who are talking in the billions, and investing in the trillions, without being able to actually back it by value.
This is while you constantly hear of people who have a say in the AI world (CEOs or investors, for example) scream at the top of their lungs āsoon AI will replace all human laborā (it wonāt). Itāll replace a lot of labor, but not all.
The reckoning hasnāt come yet, but when it comes a lot of people will be in a lot of pain. I wouldnāt be amazed if AI were to be the start of the next economic depression.
I wonder what the useful life for these chips is in AI.
We might see a phase when zillions of these chips are being upgraded, so there will be a flood of cheap secondhand HBM chips.
So we already have more AI centres than we need , in the name of national interest?
I cant see why location matters, given the internet makes all locations close.
A lot of those are afterthoughts, not new topics What is driving it is LLM. Its compute requirements are insane.
I have some genetics software that would really benefit from access to a large compute centre ā¦ I bet i cant get access ā¦ I cant even find out what OS these centres run or how to execute a program in one ā¦ all I see is llm.?
You mean financial, but there is a technical issue too. These centres will age and redund. We may get to the point where you can plug an AI centre brick into your desktop, or even your portable., and they will probably come free in your breakfast cerial packet. Then DCās will be like ancient mainframes.
I donāt believe the USA would want to run their research on an AI center in Russia. So location means a lot and who controls the AI.
That not what an AI center is about. Companies / people do not run programs on an AI. As I understand it, you query this huge database of knowledge with questions. Have you tried asking AI about an answer to your research project? Maybe give AI the specifications of your Fortran program and have it run a simulation?
It would be faster then typing in your program, but you could do that too. Type in your Fortran program into a file, post it into AI and have it run a simulation on it.
At least that my understanding. I had it write the code for color guessing game Mastermind. Simple but prove you can run simulation on AI.
Yes, it is coming. New PCās will soon have a NPUs (neural processing units) built into them for AI processing. And also there is something called open source AI that you can run on your very own 3 to 4 thousand dollars PC.
Note: Cost for the PC to run an AI given by Copilot.
Update
After more questioning to the AI, the cost could run under $1,000. for a 10 to 14 billion parameter AI. The 3 to 4,000 was for a much larger 50 to 70 billion parameter AI.
It is R, C and Fortran ā¦ yes a mixed language program. Way too big to type in and it requires an R environment.
What do you mean by run a simulation? Would it make something like a VM and run the program inside it?
OK, so it is not a general purpose computer.
That link in the first post said it takes 200 AI centres to do a training run for chatGPT .
So it is used for trainingā¦ as well as serving an LLM and a large database.
I wonder if I could use those for general computing ,ā¦ like running my program. I hope they dont make them so specialized that they are only useful for llmās?
The cost seems doable
I am still grappling with the maths of llmās ā¦ keep getting distracted ā¦ with what I have seen so far , it looks to me like they have not put thought into organising the matrix calculations ā¦ they seem to have just dived into āwe need a huge computerā.
There are usually ways of rewriting a model to simplify its calculations.
OK, we dont live in a perfect world .
I am grateful that I bought my mini-PC and an extra āusedā laptop with updated RAM and SSDs months before the RAMpocolypse. Now, Iām going to have to hold on to my computers until around 2045, when RAM prices will finally go downāha, it wonāt matter by then, weāll be dead or forced to rent our ācomputersā virtually in the cloud. I feel like Iām living in some bad dystopian novel here.
quote ā640K ought to be enough for anybodyā is an urban legend. Bill Gates has denied saying it multiple times,
If we repeatedly say it then only DSL Linux will run on that. Command line here I come
AI can not run a program. If you were to paste the code in a chat, AI could read the code and predict the output with a high degree of being correct. No execution, no VM.
That did not sound right, so I went back to the article and also search using Copilot.
The quote was āin order to train that AI, weāre looking at entire data centres, 200,000 AI chips all working togetherā
So it was one data center with 200k chips (GPUās) working together to train an AI or LLM.
No, my understanding is the NPU will be more like an attached processor to handle AI type logic. Not a replacement for the CPU inside the PC.
This way over my head. So again I turned to Copilot for an explanation. The pages of formulas that were listed was again like Greek to me. I could copy/paste the first page or 2 if you wanted to see the output shown to see.
Oh Dear ā¦ lousy reporting. We should never naively believe what we read.
200,000 GPUās no wonder there is a shortage
I have that thanks. It is my grasp of it that is lacking. I am about 1/4 way into it.
The trouble is, like all scientific stuff, they abbreviate the presentation so much that it is almost impossible to follow.
I need to compute the numbers with an example and look at each step . I am doing it in R.
The other thing I need to do is relate it back to what I know about statistical theory. They give the method, but not its justification.
I thought it might act like the GPU and be able to compute jobs handed to it by the cpu.?
Wait and see.
I think the computer that runs the AI software could run any program, if the OS allowed it.
What OS do they use in these AI centres?.. it is likely to be some form of Linux.
Well, you are right. Checking 2 AIās and also performing a search on the web, they all agreed the top 3 are:
Ubuntu Server
Red Hat Enterprise Linux (RHEL)
Rocky Linux / CentOS
The primary Linux distributions used in AI data centers, training clusters,
and supercomputers: ---- Gemini
I have not heard of Rocky Linux before.
I could hardly believe that number. 200k GPU acting as a single computer! How in the hell does someone program such a monster not to mention having them all wired together.
You can, currently, program any Linux to send parts of numerical calculations to a GPUā¦ typically matrix arithmetic. The software for doing it is CUDA for nvidia GPUās and OpenCL for AMD GPUās.
@daniel.m.tripp has done it for running ffmpeg.
But
I cant imagine one CPU controlling 200K GPUās ā¦ it would be too busy.
They must split the load somehow and have multiple CPUās each contolling maybe 10 GPUās
So they want battle-ready server versions.
I wonder whyā¦ thats the last thing a scientist would want ā¦ all that crappy overhead for what.?
Rocky Linux was created to replace CentOS when Red Hat changed the purpose of CentOS.
Here is the Copilot summary:
Rocky Linux exists to provide a stable, free, communityāmaintained enterprise Linux that behaves like traditional CentOS used to. Its purpose is to offer a longāterm, predictable RHELācompatible OS for servers, businesses, and anyone who needs rockāsolid stability without subscription fees.
CentOS, before its shift in 2020, was a rebuild of Red Hat Enterprise Linux that offered the same stability and behavior but without the cost. It was widely used in production environments because it was essentially RHEL minus the branding. When Red Hat repositioned CentOS into the fasterāmoving CentOS Stream, Rocky Linux was created to fill the gap left behind.