AI Tokens: See How They Are Derived From Scratch

Setting Up a Safe, Clean Sandbox

Note: Checked with moderator and received permission to post.

I’m starting a series of posts showing the creation of a local, character-level Transformer model to see how tokens are derived from scratch.

Full disclosure: This isn’t going to be my code! I do not know how to code in python. I asked Gemini to put together a hands-on learning project to break down and show how tokens work. To keep from having a posting that is too long, I will post the different steps that I went through as I did them.

The first step was about setting up the sandbox.

AI suggested this environment setup, and it sounded good to me—plus, Linux Mint needed it to keep things stable. It gave me an isolated environment to work in and keeps me from loading libraries into root that I might not need in the future. To avoid that, the setup does two things:

I set up a separate partition to keep the main drive clean.

It used a Python Virtual Environment (venv), which acts like a self-contained bubble.

The Setup Commands

First, I opened the terminal and navigated to the the directory:

cd /media/easyt/ALVM

Next, I started the virtual environment bubble (I called it ai_env) by entering:

python3 -m venv ai_env

Finally, I activated it. I knew I was in the bubble when the terminal prompt changed to show (ai_env) at the front:

source ai_env/bin/activate

The initial sandbox setup completed the first step for me. Next some code.

4 Likes

Tokens - continued.

Next I needed a text file for the tokenization. I choose the ‘Fables of Aesop’ and places it in a regular text (.txt) file (fables.txt). The text file is about 65k in size. The first 10 lines of the file looks like this;

Title: The Fables of Aesop
Author: Aesop

Contents

The Cock and the Pearl
The Wolf and the Lamb
The Dog and the Shadow
The Lion’s Share
The Wolf and the Crane

This post is taking the first real code step toward turning that raw text into math: building the vocabulary.

Because AI is making a character-level model, our vocabulary isn’t a massive dictionary of full words. It is simply a list of every single unique character that appears anywhere in our text file—letters (uppercase and lowercase), punctuation, numbers, and spaces.

The Python Code was places in a file called build_vocab.py. Code generated by AI.

Python


#  Open and read the raw text file
with open('fables.txt', 'r', encoding='utf-8') as f:
    text = f.read()

#  Get all unique characters and sort them
chars = sorted(list(set(text)))
vocab_size = len(chars)

#  Print out what we found
print("Unique characters found:", "".join(chars))
print(f"Vocabulary Size: {vocab_size}")

I ran the script. I also remove the ‘#’ form the comments in the code above to remove bold printing. Replace with ‘C’.

python3 build_vocab.py

The Python code scanned all the text in fables.txt, threw out all duplicates using set(), sorted what was left, and printed out the total count.

For my dataset, it found 65 unique characters (including spaces, periods, quotes, and newlines). The entire “vocabulary” for this model is only about 65 items.

Copy of terminal

easyt@E14:~$ cd /media/easyt/ALVM
easyt@E14:/media/easyt/ALVM$ python3 -m venv ai_env
easyt@E14:/media/easyt/ALVM$ source ai_env/bin/activate
(ai_env) easyt@E14:/media/easyt/ALVM$ python3 build_vocab.py
Unique characters found:
!,-.:;?ABCDEFGHIJKLMNOPQRSTUVWY_abcdefghijklmnopqrstuvwxyzÆ—’“”
Vocabulary Size: 65

Note, no cap Z was found.

5 Likes

I assume you are now going to use that set of characters to generate a model matrix?
It should only be a 65 x 65 matrix … that will be workable.

1 Like

I think you can put the code in a post and include the comments as they were if you paste it into a preformatted section. Like the following:

# /// script
# requires-python = "==3.14"
# dependencies = [
#     "aiohttp",
#     "beautifulsoup4",
#     "Brotli",
# ]
# ///

import asyncio
import aiohttp
from bs4 import BeautifulSoup
from datetime import datetime
from urllib.parse import urljoin, urlparse
import logging
import re
import brotli  # noqa: F401

# Configure logging
timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")

I clicked the little </> shaped button on the toolbar while creating the post. That created the preformatted text section to paste into.

4 Likes

Thanks @pdecker, I knew about quote but didn’t know about </>.

3 Likes

You can also type three backquotes on a separate line
then copy in your code text
then end with another three backquotes on separate line

That puts the text in a markdown code box

4 Likes

Put all your Python lines and computer output in boxes … it makes it easier to read
Do you want me to edit it to show what I mean?

1 Like

Yes, I am willing to learn a new way of posting.

1 Like

Will do. I aill use a message
I have made a mess … your reply no 6 is now a private message to you.
I will try and move to back to here, but if I fail , would you mind posting it again.
It contains my edits … to see how I did it go into edit mode.
Sorry
Neville

1 Like

The next step was to show how many unique charterers there were in the fables.txt and build a vocabulary.

python s2-find-uq.py
Total characters in book: 63497
Unique characters (vocabulary size): 65
First 10 characters in vocabulary: ['\n', ' ', '!', ',', '-', '.', ':', ';', '?', 'A']

Computers can not process letters; they process numbers. To give our neural network something it can calculate, every single character in the vocabulary gets assigned its own number (unique integer ID).

Next I needed to apply an encoder to the entire dataset and turning it into a PyTorch Tensor.

A PyTorch Tensor is basically a multi-dimensional array designed specifically for high-speed mathematical operations on neural networks.

Installing PyTorch

Before I could run the script, I needed to make sure PyTorch was installed inside my virtual environment (ai_env). In my terminal, I ran:

pip install torch

The AI then created a new python script called s4-train1.py inside my project folder to tokenize the entire text file and store it as a Tensor:

The Python Code

import torch
# Step 1: Open and read your fables book
with open("fables.txt", "r", encoding="utf-8") as f:
    text = f.read()

# Step 2: Find all unique characters in the book
chars = sorted(list(set(text)))
vocab_size = len(chars)

# Step 3: Create mappings (character to number, and number to character)
char_to_int = { ch:i for i,ch in enumerate(chars) }
int_to_char = { i:ch for i,ch in enumerate(chars) }

# Step 4: Print our results to see what the AI's "alphabet" looks like
print(f"Total characters in book: {len(text)}")
print(f"Unique characters (vocabulary size): {vocab_size}")
print(f"First 10 characters in vocabulary: {chars[:10]}")

# Step 5: Encode the entire book into numbers
data = torch.tensor([char_to_int[c] for c in text], dtype=torch.long)

# Step 6: Let's see what the first 100 characters look like as numbers
print("\nFirst 100 characters as text:")
print(text[:100])

print("\nFirst 100 characters as numbers:")
print(data[:100])

# Step 7: Split data into training and validation sets
n = int(0.9 * len(data)) # 90% of the total characters
train_data = data[:n]
val_data = data[n:]

print(f"\nTraining set size: {len(train_data)} characters")
print(f"Validation set size: {len(val_data)} characters")

Inside my activated ai_env terminal, I ran the python script:

python s4-train1.py

What Happened

Python took the entire fables file, translated every single character into its integer ID, and stored it inside a single 1D Tensor array.

When printed, the terminal showed something like:
(output of all the steps)

 python s4-train1.py
Total characters in book: 63497
Unique characters (vocabulary size): 65
First 10 characters in vocabulary: ['\n', ' ', '!', ',', '-', '.', ':', ';', '?', 'A']

First 100 characters as text:

Title: The Fables of Aesop
Author: Aesop

Contents

 The Cock and the Pearl
 The Wolf and the Lamb

First 100 characters as numbers:

tensor([28, 42, 53, 45, 38,  6,  1, 28, 41, 38,  1, 14, 34, 35, 45, 38, 52, 1, 
48, 39,  1,  9, 38, 52, 48, 49,  0,  9, 54, 53, 41, 48, 51,  6,  1, 9, 38, 52, 
48, 49,  0,  0, 11, 48, 47, 53, 38, 47, 53, 52,  0,  0,  1, 28, 41, 38,  1, 11, 
48, 36, 44,  1, 34, 47, 37,  1, 53, 41, 38,  1, 24, 38, 34, 51, 45,  0,  1, 28, 
41, 38,  1, 31, 48, 45, 39,  1, 34, 47, 37, 1, 53, 41, 38,  1, 20, 34, 46, 35,
  0,  1])

Training set size: 57147 characters
Validation set size: 6350 characters

The entire text file is now officially converted into a clean stream of numbers that PyTorch can feed straight into an AI model! All 57,147 numbers.

Next posting will show training chunks and set up the train/validation.

2 Likes

I’m documenting my process of building a tiny Transformer language model from scratch using PyTorch. All formatting and wording are assisted by AI, so was the code for the python scripts (programs).

This series walks through:
turning raw text into tensors
splitting data into training/validation sets
building a Bigram model
adding Self‑Attention
scaling to Multi‑Head Attention
assembling a full Transformer Block

1. Train/Validation Split + Context Blocks

After loading the dataset into a single giant PyTorch tensor, I split it:
Training set: 90%
Validation set: 10%

Output from script:

Preformatted text`Training set size: 57147 characters
Validation set size: 6350 characterstype or paste code here

The model learns by looking at a fixed window of tokens (e.g., 8 tokens) and predicting the next one. Example predictions:
Output from script:

'T'      → 'i'
'Ti'     → 't'
'Tit'    → 'l'
'Title:' → ' '
'Title: '→ 'T'

Each 8‑token block produces 8 training examples, one per position.

2. Bigram Language Model

A Bigram model is the simplest possible language model:
Given character X, predict the next character Y.

Output from script:

Inputs shape:  torch.Size([4, 8])
Targets shape: torch.Size([4, 8])
Logits shape:  torch.Size([32, 65])
Initial Loss:  4.5218

32 tokens (4 batches × 8 tokens)
65 logits per token (one per vocabulary character)
Loss ~4.5 → basically random guessing

Training

After adding AdamW and training for 10,000 steps:
Output from script:

Final Training Loss: 2.2108

Generated text:

V” he uthil  acey Bala
sarathe asssef tCay wnge hedor s?u t.ADjKYTh Ag:
 athes me Foom tsaged fo q’?Dmbyr,”shhe the usilo te the t isbut bbed nes is t ry *he rthiove Moxpegoa;asus wor t Glithrout I IJDng
...

It’s gibberish, but it learned:
spaces, short English words (the, he, me, is), basic character patterns

A Bigram model can’t form real sentences because it only sees one character of context.

3. Introducing Self‑Attention

To produce meaningful text, the model must look back at multiple previous tokens, not just one.

Self‑Attention gives each token three vectors:

Query: what am I looking for?
Key: what do I contain?
Value: what information do I provide if I’m relevant?

The head computes dot‑products between Queries and Keys, applies a causal mask, and aggregates Values.

Output from script:

Input shape:  torch.Size([4, 8, 32])
Output shape: torch.Size([4, 8, 16])

A single head learns one type of relationship (punctuation, vowels, etc.).
To learn multiple relationships simultaneously, we need Multi‑Head Attention.

4. Multi‑Head Attention + Transformer Block

A Transformer Block combines:

Multi‑Head Attention
Feed‑Forward Network (FFN)
Residual connections
Layer normalization

This preserves shape:
Output from script:

Input:  torch.Size([4, 8, 32])
Output: torch.Size([4, 8, 32])

Training
Output from script:

Step 0 | Loss: 4.2122
Step 1000 | Loss: 1.8939

Final Step Loss: 1.8307

Output from script:
Generated text:

said decler Pits sablye, with samouliced beand the cander, should Boy weall pittleverile wereawill.”
...

It’s still nonsense, but noticeably more structured

longer word patterns, repeated motifs, and primitive sentence‑like shapes

This confirms the Transformer Block is functioning.

Conclusion
I went thru the steps and built

a tokenizer
a Bigram model
a Self‑Attention head
a Multi‑Head Attention
a full Transformer Block

I will next post the entire output form the script the AI generated and the full script.
This also completes my series on learning about tokens.

3 Likes

Full output form the last run of the python script.

Total characters in book: 63497
Unique characters (vocabulary size): 65
First 10 characters in vocabulary: ['\n', ' ', '!', ',', '-', '.', ':', ';', '?', 'A']

First 100 characters as text:
Title: The Fables of Aesop
Author: Aesop

Contents

 The Cock and the Pearl
 The Wolf and the Lamb
 

First 100 characters as numbers:
tensor([28, 42, 53, 45, 38,  6,  1, 28, 41, 38,  1, 14, 34, 35, 45, 38, 52,  1,
        48, 39,  1,  9, 38, 52, 48, 49,  0,  9, 54, 53, 41, 48, 51,  6,  1,  9,
        38, 52, 48, 49,  0,  0, 11, 48, 47, 53, 38, 47, 53, 52,  0,  0,  1, 28,
        41, 38,  1, 11, 48, 36, 44,  1, 34, 47, 37,  1, 53, 41, 38,  1, 24, 38,
        34, 51, 45,  0,  1, 28, 41, 38,  1, 31, 48, 45, 39,  1, 34, 47, 37,  1,
        53, 41, 38,  1, 20, 34, 46, 35,  0,  1])

Training set size: 57147 characters
Validation set size: 6350 characters

--- AI Prediction Examples ---
When input is: 'T' ---> Predict target: 'i'
When input is: 'Ti' ---> Predict target: 't'
When input is: 'Tit' ---> Predict target: 'l'
When input is: 'Titl' ---> Predict target: 'e'
When input is: 'Title' ---> Predict target: ':'
When input is: 'Title:' ---> Predict target: ' '
When input is: 'Title: ' ---> Predict target: 'T'
When input is: 'Title: T' ---> Predict target: 'h'

--- Batch Details ---
Inputs shape (batch_size, block_size): torch.Size([4, 8])
Targets shape (batch_size, block_size): torch.Size([4, 8])

Inputs matrix:
tensor([[38,  1, 35, 58, 52, 53, 34, 47],
        [ 1, 34, 52,  1, 53, 41, 34, 53],
        [47, 37,  1, 34, 47, 37,  1, 53],
        [ 0,  0,  0,  0,  0, 28, 41, 38]])

--- Deep Transformer Training Started ---
Step     0 | Loss: 4.2122
Step  1000 | Loss: 1.8939
Step  2000 | Loss: 2.4123
Step  3000 | Loss: 2.4483
Step  4000 | Loss: 1.9745
Step  5000 | Loss: 1.8912
Step  6000 | Loss: 1.8226
Step  7000 | Loss: 2.4721
Step  8000 | Loss: 1.8434
Step  9000 | Loss: 1.7932
Final Step Loss: 1.8307

--- AI Generated Text ---
said decler Pits sablye, with samouliced beand the cander, should Boy weall pittleverile wereawill.” said to the stray and capaming them, Wook and his said:
his spose what and town and thim the Cith. I arm aftime if dearplect ittlouts that Whome man in as could try
whid see Qordoy’h samed and toll m

2 Likes

The AI generated code for the script.

import torch

# Step 1: Open and read your fables book
with open("fables.txt", "r", encoding="utf-8") as f:
    text = f.read()

# Step 2: Find all unique characters in the book
chars = sorted(list(set(text)))
vocab_size = len(chars)

# Step 3: Create mappings (character to number, and number to character)
char_to_int = { ch:i for i,ch in enumerate(chars) }
int_to_char = { i:ch for i,ch in enumerate(chars) }

# Step 4: Print our results to see what the AI's "alphabet" looks like
print(f"Total characters in book: {len(text)}")
print(f"Unique characters (vocabulary size): {vocab_size}")
print(f"First 10 characters in vocabulary: {chars[:10]}")

# Step 5: Encode the entire book into numbers
data = torch.tensor([char_to_int[c] for c in text], dtype=torch.long)

# Step 6: Let's see what the first 100 characters look like as numbers
print("\nFirst 100 characters as text:")
print(text[:100])

print("\nFirst 100 characters as numbers:")
print(data[:100])

# Step 7: Split data into training and validation sets
n = int(0.9 * len(data)) # 90% of the total characters
train_data = data[:n]
val_data = data[n:]

print(f"\nTraining set size: {len(train_data)} characters")
print(f"Validation set size: {len(val_data)} characters")

# Step 8: Show how the AI learns from a single block of text
block_size = 8
x = train_data[:block_size]      # The input to the AI
y = train_data[1:block_size+1]    # The target (what we want it to predict)

print("\n--- AI Prediction Examples ---")
for t in range(block_size):
    context = x[:t+1]
    target = y[t]
    # We turn the numbers back to letters so we can read it easily
    context_str = "".join([int_to_char[int(i)] for i in context])
    target_str = int_to_char[int(target)]
    print(f"When input is: {repr(context_str)} ---> Predict target: {repr(target_str)}")

# Step 9: Create a function to pull random batches of data
torch.manual_seed(1337) # Keep things reproducible
batch_size = 4 # How many sequences to process at once
block_size = 8 # Length of each sequence

def get_batch(split):
    # Select which dataset to pull from
    data_set = train_data if split == 'train' else val_data
    # Pick random starting indexes in the data
    ix = torch.randint(len(data_set) - block_size, (batch_size,))
    # Stack the inputs (x) and targets (y) into matrices
    x = torch.stack([data_set[i:i+block_size] for i in ix])
    y = torch.stack([data_set[i+1:i+block_size+1] for i in ix])
    return x, y

# Pull a quick training batch to inspect
xb, yb = get_batch('train')
print("\n--- Batch Details ---")
print("Inputs shape (batch_size, block_size):", xb.shape)
print("Targets shape (batch_size, block_size):", yb.shape)
print("\nInputs matrix:")
print(xb)

import torch.nn as nn
from torch.nn import functional as F

# Step 10: Multi-Head Attention and Transformer Blocks
class Head(nn.Module):
    """ One head of self-attention """
    def __init__(self, head_size):
        super().__init__()
        self.key = nn.Linear(32, head_size, bias=False)
        self.query = nn.Linear(32, head_size, bias=False)
        self.value = nn.Linear(32, head_size, bias=False)
        self.register_buffer('tril', torch.tril(torch.ones(block_size, block_size)))

    def forward(self, x):
        B, T, C = x.shape
        k = self.key(x)   
        q = self.query(x) 
        wei = q @ k.transpose(-2, -1) * (k.shape[-1]**-0.5)
        wei = wei.masked_fill(self.tril[:T, :T] == 0, float('-inf'))
        wei = F.softmax(wei, dim=-1)
        v = self.value(x) 
        out = wei @ v 
        return out

class MultiHeadAttention(nn.Module):
    """ Multiple heads of self-attention running in parallel """
    def __init__(self, num_heads, head_size):
        super().__init__()
        self.heads = nn.ModuleList([Head(head_size) for _ in range(num_heads)])
        self.proj = nn.Linear(32, 32) # Project back to original dimension

    def forward(self, x):
        # Concatenate the outputs of all heads together
        out = torch.cat([h(x) for h in self.heads], dim=-1)
        out = self.proj(out)
        return out

class FeedForward(nn.Module):
    """ A simple linear layer followed by a non-linearity """
    def __init__(self, n_embd):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(n_embd, 4 * n_embd),
            nn.ReLU(),
            nn.Linear(4 * n_embd, n_embd),
        )

    def forward(self, x):
        return self.net(x)

class Block(nn.Module):
    """ Transformer Block: communication (attention) followed by computation (feed-forward) """
    def __init__(self, n_embd, n_head):
        super().__init__()
        head_size = n_embd // n_head
        self.sa = MultiHeadAttention(n_head, head_size)
        self.ffwd = FeedForward(n_embd)
        self.ln1 = nn.LayerNorm(n_embd)
        self.ln2 = nn.LayerNorm(n_embd)

    def forward(self, x):
        # Residual connections (skips) and Layer Normalization
        x = x + self.sa(self.ln1(x))
        x = x + self.ffwd(self.ln2(x))
        return x

# Step 11: The Complete Transformer Model
class TransformerLanguageModel(nn.Module):
    def __init__(self, vocab_size):
        super().__init__()
        self.token_embedding_table = nn.Embedding(vocab_size, 32)
        self.position_embedding_table = nn.Embedding(block_size, 32)
        # Stack 3 Transformer Blocks!
        self.blocks = nn.Sequential(
            Block(32, n_head=4),
            Block(32, n_head=4),
            Block(32, n_head=4),
        )
        self.ln_f = nn.LayerNorm(32) # Final layer norm
        self.lm_head = nn.Linear(32, vocab_size)

    def forward(self, idx, targets=None):
        B, T = idx.shape
        tok_emb = self.token_embedding_table(idx) 
        pos_emb = self.position_embedding_table(torch.arange(T, device=idx.device)) 
        x = tok_emb + pos_emb 
        x = self.blocks(x)
        x = self.ln_f(x)
        logits = self.lm_head(x) 
        
        if targets is None:
            loss = None
        else:
            B, T, C = logits.shape
            logits = logits.view(B*T, C)
            targets = targets.view(B*T)
            loss = F.cross_entropy(logits, targets)
        return logits, loss

# Initialize our deep model
model = TransformerLanguageModel(vocab_size)

# Step 12: Train the Transformer
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)

print("\n--- Deep Transformer Training Started ---")
for steps in range(10000):
    xb, yb = get_batch('train')
    logits, loss = model(xb, yb)
    
    optimizer.zero_grad(set_to_none=True)
    loss.backward()
    optimizer.step()
    
    if steps % 1000 == 0:
        print(f"Step {steps:5d} | Loss: {loss.item():.4f}")

print(f"Final Step Loss: {loss.item():.4f}")

# Step 13: Generate Text
def generate_text(model, max_new_tokens=200):
    context_tensor = torch.zeros((1, 1), dtype=torch.long)
    generated_indices = []
    for _ in range(max_new_tokens):
        context_cond = context_tensor[:, -block_size:]
        logits, _ = model(context_cond)
        logits = logits[:, -1, :] 
        probs = F.softmax(logits, dim=-1)
        next_token = torch.multinomial(probs, num_samples=1)
        generated_indices.append(next_token.item())
        context_tensor = torch.cat((context_tensor, next_token), dim=1)
    return "".join([int_to_char[idx] for idx in generated_indices])

print("\n--- AI Generated Text ---")
print(generate_text(model, max_new_tokens=300))
3 Likes

I get some of it, but my understanding is not full.
I think I need to run it myself.
The style of programming is difficult for me.

3 Likes

I found it very interesting and even some fun to go thru the phases of building a tiny AI. You most likely would understand the math behind the models.

4 Likes

I would like to get to that point.

2 Likes

Watching a “Baby” AI Learn to Speak

I thought I would try to compare my tiny AI to that of a baby learning to speak. With the tiny AI that Gemini built for this learning experience for me, the initial output was not impressive at all—just gibberish. But isn’t that what you would expect from maybe a baby at 1 year old or less?

I was not trying to build a full-fledged AI like Gemini or ChatGPT, which would be impossible on my PC anyway. Instead, I am trying to learn how these massive AIs learn.

When building a tiny, character-level (looking at only one character at a time) Transformer from scratch on a CPU, watching the raw output is a lot like watching a toddler learn to speak. It starts with random babbling and slowly progresses to real words as you expand its “brain” capacity and give it better memory.

The Control Knobs: What Powers the Tiny AI Brain

Before looking at the growth stages, here are the 5 main “knobs” we can turn in the code, defined in plain English:

n_embd (Embedding Dimension) — Brain Detail & Capacity

What it is: How many numbers the AI uses to represent each character’s identity and meaning.

Impact on output: A small value (32) gives the AI a blurry picture of language. Bumping it to 64 gives the AI a sharper internal dictionary, letting it distinguish subtle patterns between letters.

block_size (Context Window) — Short-Term Memory

What it is: How many characters the AI can look back at when deciding what to write next. It can only see 8 characters at a time.

Impact on output: If block_size is only 8, the AI forgets how a word or sentence started before it finishes writing it. Increasing it to 32 gives it enough memory to complete full words and basic 3-to-4 word phrases.

temperature — Creativity vs. Focus

What it is: A dial controlling how “wildly” or “cautiously” the AI picks its next character.

Impact on output: High temperature makes the AI take wild, unpredictable guesses (gibberish). Lower temperature (0.5) cools it down, forcing it to pick safer, highly likely characters.

top_k — The Filter / Guardrails

What it is: Limits the AI’s choices to only the top K most likely next letters, throwing out all the weird bottom choices.

Impact on output: Setting top_k = 3 acts like speech therapy—it forces the AI to pick only from its top 3 best guesses, instantly eliminating bizarre character combinations.

batch_size & Dataset Size — Study Material & Study Pace

What it is: Dataset size is the total volume of text (books) given to the AI; batch_size is how many chunks of text it processes in a single study step.

Impact on output: Feeding it just 63k characters (Aesop’s Fables) is like giving a child a single thin picture book. Expanding the dataset to ~500k+ characters (fables + fairy tales) provides the statistical depth needed to learn real spelling rules.

The Growth Stages

  1. The Babble Stage (6 Months to 1 Year)

Configuration: block_size = 8 | n_embd = 32 | High/Unfiltered Temperature

What’s happening: The model only looks back 8 characters at a time and has a tiny internal map (32). Without temperature scaling or top-k bounds, it takes wild guesses at the next character.

Output: Random character strings and broken syllables.

MY input to the tiny AI from my last posting: fox
AI completed your story:
Code

-> fox spried come Pits sablye, with samouliced beand the cander, should Boy weall
  1. The First Words Stage (1 to 2 Years)

Configuration: block_size = 8 | n_embd = 64 | temperature = 0.5 | top_k = 3

What’s happening: Doubling the embedding size (64) gives the model a richer internal map. Setting temperature to 0.5 and top-k to 3 acts like speech therapy—it forces the AI to choose only from its most confident options.

Output: Recognizable English words appear, but grammar remains chaotic because an 8-character memory window is too short to remember how a sentence started.

My input to the tiny AI: the fox and the
AI completed your story:
Code

-> the fox and the Fox he to he straces the Fox the Frog a Fox the Wolf a Fox and and have and the Fox and the Fox and and the Frog and had his he had to his all he have

My input to the tiny AI: a hungry lion
AI completed your story:
Code

-> a hungry lion a her, but and stook the Fox and the Wolf his have the Wolf have the Wolf he old he old his that a looking a Fox a purer happenter. “The Fox and the F

What’s Next?

For additional fun and to make the tiny AI grow up a bit, Gemini has suggested giving the AI a larger text file to learn from and to increase the size of the parameters.

To grow for the AI it need to go from looking at just one character at a time to two or more characters at a time (subword tokens).

I will update the AI provided script and post back the results of these new learning phases in my next post!

In Summary of What Impacts an AI the Most

  • Temperature & Top-K = Instant Speech Clean-Up: Muting unlikely guesses immediately cuts out the gibberish without needing a larger model.
  • Context Window (block_size) = Sentence Structure: Short memory windows cause word repetitions (like “with the with”). Wider windows allow attention mechanisms to track subjects across phrases.
  • Dataset Size = Vocabulary & Grammar: A 63k-character dataset is like giving a child a single thin picture book. Expanding to ~500k characters provides enough statistical repetition to learn actual English spelling patterns letter-by-letter.

These posting are AI driven and used as a learning tool.

4 Likes

Well done Howard.
The analogy with a baby learning to speak is fascinating.

3 Likes