on the merits of denying intelligence

Thanks for the thoughtful comment, haha. Without getting too much into the weeds, I think : Each of our perspectives on what AGI / or "I" in general means is shaped by unique personal experience, a bound set for each of us. These sets may or may not be commensurable - we don't currently know if the categories or your experience, and the categories of mine are even comparable. This has political consequences, mainly related to how seriously we take each other's speech.

Some of the technological steps you mentioned however, I must say are "operational improvements" which improve cache hierarchy and introduce compression in some cases -without really changing the overall "architectural" approach employed by the currently trendy transformer gang. 

I think it remains useful to query both, other humans, and the results of what we can make machines do ... in improving not just "what we think intelligence is", but also how we can communicate what we think it is.

the ambiguity of knowledge

A parent asked, if others had asked their children, what their children wanted from education - and how to judge if the child's preferences or the parents' preferences were better.

I elected to share a canonical rant : before I went to Year 1 of elementary school, my parents asked if I wanted to study in a school with two, or three languages. I said two, because I wanted to be lazy. They sent me to the school with three. So for the rest of my life, I have been wary of fools asking meaningless questions.

Aside from that, of course life has provided many disparate experiences. Many have involved speaking with people who tell me, nicely or otherwise, that they believe I am not well-informed or well-opinionated. The problem with me, I find, is that while I am quite willing to accept these views, and submit to compromise, more often than not, I find that I should have paid less attention to them, because the speaker was foolish. I can only speak from my own experience, which is quite limited.

I do not believe this makes me intelligent, but sometimes intelligence is something you have whether you want to have it or not. And others lack it, whether you wish they had it or not.

And sometimes you are just stupid. :)

maths has an ISO notation

Day 1 back at elementary number theory ( day 1155 of sabbatical2, day ~15,547 on earth ). I am revising this slide on the notation for exponentiation and logarithms which I made year ago.

• after a bunch of catch up on software engineering, I am finally turning to focus on remediating my poor foundations in number theory
• a quick look around the internet will show you that this "has always been, and continues to be" an issue for many students of maths; folks also decry other notations like dx/dy, and the visual non-symmetry of notation for differentiation and integration
• I think, a pet idea of mine henceforth will be  : someone should make an ISO standard for canonical mathematical notation, and then republish all past mathematical texts ( or have a way of translating them automatically ) to the canonical mathematical notation; all mathematicians would be of course free to use non-ISO notation anywhere they liked. ( oh wait, they've been doing it since 1992, and the latest is here : https://en.wikipedia.org/wiki/ISO/IEC_80000 )

TIL :(or am reminded, because I probably forgot) : 

• (1) there's an ISO for maths notation, so speculation about notation optimisation should just target that, 
• (2) the notion of "antilogs" we studied in highschool are already deprecated in scientific writing, 
• (3) the etymology of "logbooks" is the same as that of "knots"

orang kebencian

Jenis orang yang paling saya benci, bukan tertumpu kepada mana-mana negara, atau kaum, walaupun ada banyak jenis ini persekelilingan hidup saya. 

Iaitu sesiapa yang dahpun dorang diajar sesuatu oleh ibubapa atau gurunya, dorang ingat pula orang lain kena ikut cara itu. Bila tak ikut pula orang lain, jenis orang yang saya paling benci akan rasa gelabah. Dorang seakan-akan petu ... yang mat saleh panggil karen. 

"Kan, saya sudah buat apa yang disuruh, kenapa orang lain tidak menjunjung perbuatanku? Mana pengiktirafan? Aku mahu respek! Cara saya mulia! Saya seorang yang berjalan lurus! KORANG, tak beradab!" 

Karen puteh ada, karen cina ada, mana-mana jenis karen pun ada. Tapi ini kerangka mengenali penyakit itu, secara am.

Saya tidak sebut pasal kemahuan seseorang. Tapi pasal orang yang kemahuannya, diterima dari ajaran, dan bukan dikerjakan sendiri.

electrolytic muscle function, and disease

At rest : 

• [ neuron and muscle independently : sodium-potassium pump (slow) : pushes out 3Na+ for every 2K+ pulled in, maintaining negative voltage inside the cell ]

Firing : 

• [ neuron : ( due to upstream stimulation) opens sodium gate, Na+ inflow sends current to calcium gate, Ca2+ enters neuron, triggers acetylcholine release into synaptic cleft ] 
• -> [ muscle : acetylcholine from synaptic cleft, opens sodium gate, Na+ inflow sends current to sarcoplasmic reticulum, which releases calcium, which causes contraction of the cell : (fast) K+ leaves cell to restore negative voltage inside the cell ]

Muscle Diseases, and ordinary states which become diseases if not reversed within ordinary timeframes :

• - Ca2+ : 
• too much outside cells, "overshields" sodium gates of BOTH neurons and muscles, both become "understimulated" ... and subsequent to neuron stimulation, Ca2+ enters neuron slower, dumping acetylcholine slower, furthering muscle "understimulation"; 
• too little outside cells, "undershields" sodium gates of BOTH neurons and muscles, both become "overstimulated" ... and subsequent to neuron stimulation, Ca2+ enters neuron faster, dumping acetylcholine faster, furthering muscle "overstimulation"
• - Mg2+ :
• too much outside cells, block Ca2+ entry to neurons, reducing acetylcholine release, "understimulating" muscles
• too little outside cells, Ca2+ entry is less blocked, increasing acetylcholine release, "overstimulating" muscles
• - K+ : 
• too much outside cell, K+ cannot leave quickly, cell may become "overcontracted" - ditto "overexcitation" of neuron at early stages, but  at later stages, sodium gates fail, neurons and muscles both weaken; 
• too little outside cell, K+ leaves too fast, cell may become "hard to contract"
• - Na+ : 
• too much outside cell, Na+ enters quickly, cell may become "overexcited", ditto for neuron; 
• too little outside cell, Na+ cannot enter quickly, "underexciting" cell, ditto for neuron; nervous system may overcompensate with erratic voltages, causing "erratic excitation"

The Cartesian Product Bubble

I'm guilty of avoiding AI over the past decades because the techniques I've heard about still seem rather primitive. The last decade in particular has been characterised by VCs throwing obscene amounts of money at a fundamentally inefficient approach.

After noodling around in the current tech for a week or so, just to make sure I understand what's going on, I think it's safe to say that the the titular concern of this post stands firm.

An analogy. At the heart of trendy LLMs is a giant Y times Y list of known words in English ( also other languages, but nevermind those for now ), forming an enormous 2D table. It's Y times Y because both H and V axes have the same list of words - sure, half the table is redundant, so you can think of it as a trangular half-rectangle of unique pairs. If you lookup the junction between any two words, you find a WINDOW to a realm of many, about 10^(3 +/- 1 ), dimensions of information about each pair of words.

Based on the "best" available public information, this is the data structure for storing everything the LLM knows about the world - which it knows ONLY FROM READING TEXT ( except for MLLMs, which we'll also ignore for now ). Even in the case of MLLMs, sensory spatio-temporal data can be understood to be stored within the realm accessed by each WINDOW.

Of course, when you ask an LLM a question like "What is the meaning of life?" it doesn't need to peek through ALL the WINDOWS and all the realms above, rather it only has to peek through a subset of windows, W_n. However it's still fundamentally inefficient, because : it doesn't JUST look at W_n and answer your question. 

Oh no.

It looks at your question, goes to a set of windows, W_n1, takes a peek, grimaces in deep thought, spits out ONE word, then pats itself on the head and goes to W_n2, a COMPLETELY DIFFERENT set of windows, spits out a SECOND word, and goes on and on until W_nM to obtain M words. While there is a cache, this is exactly what it sounds like - quite a bit of work.

I'm not a very smart fellow, so I'm probably going to be wrong about this. But, it would seem that the thin red circle that I am drawing around the entire architecture that drives most of our current AI tech is going to have to collapse, and be replaced by more efficient methods soon enough.

When? Who knows. I've been waiting since 2003, and "they" haven't figured it out yet. Some of "them" are pretty close, I think. I like the JEPA and MLLM approaches, and am eager to see where they end up.

One thing's for sure - if you create a model of the world based on this Cartesian product approach to windows upon a realm of thousands of dimensions, and have to crawl through the whole library, to peek through a different SET of windows, once per answer word ... even if it gets you the correct answer to a question, it's bloody tiring.

And to BUILD that library of windows to that realm ... it still takes WAY more money than it takes to build a human brain.