this post was submitted on 15 Sep 2026
18 points (64.1% liked)

Technology

88051 readers
3189 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
18
Someone built a visualization of what 1 million tokens actually looks like (www.1millioncontext.com)
submitted 2 days ago by khasky@lemmy.world to c/technology@lemmy.world
5 comments fedilink hide all child comments
 

I found One Million Tokens. Its rough scale:

1M tokens
~ 750K words
~ 3,000 pages
~ 83 hours of conversation
~ 75K lines of code

🧠 The more interesting part is the timeline.

It starts with GPT-3 at 2,048 tokens in 2020, then walks through ChatGPT 4K, GPT-4 32K, Claude 100K, Gemini 1M, and the later multi-million-token era.

The visual change is kind of absurd when you see all the pages stacked together.

One caveat: maximum context is not the same as perfect memory/retrieval. A model accepting 1M tokens can still fail to use information buried inside that window.

The line-of-code/page conversions are approximations too. I am curious how people here design around 1M+ context in practice.

Do you actually feed giant source sets directly, or still prefer retrieval + smaller focused contexts for cost/attention reasons?

you are viewing a single comment's thread
view the rest of the comments
[–] tyler@programming.dev 1 points 3 hours ago

In what world is a production codebase 75k lines. A single project at my last company was 1.4 million lines of code. Every single one of our microservices had more lines of code than that. And just my team was responsible for 50+ microservices. And thirteen tokens per line is hilariously low.

Holding a week of chat in your head is nothing when 90% of it is stuff that is incorrect or discussed and minds changed. This is exactly why my boss keeps saying conflicting things, the LLM doesn’t know what is true or not. It just regurgitates the random thing from context.