Today, I'm reading philosophy. A classic issue is "the problem of universals" ( summary : what makes the word chair, apply to this thing, and not that thing? ).
So, it reminds me that this, "leaky abstraction", is at the heart of problems around "governance failure" in current LLMs, and with people in general (e.g., "never do X" or "always do Y" ).
It's nice nowadays, to be able to take ancient fuzzy concepts and apply mechanistic interpretation to them!
Of late, Md Ismail Sojal has been one of the most prolific, coherent, Facebook content curators, on topics related to artificial intelligence. Today he posted a brief summary of Demis Hassabis' career, which caught my attention. I took particular interest in learning that Hassabis published important papers on the hippocampus, from 2007 onwards, in an area of my research : delineating the software topology of human intelligence. Given how old the paper is, I've probably been influenced by it in ways immemorial. Its findings align very well with my own practical study of how to quantify my conscious experience on moment to moment basis, something I've been studying since around 2003.
In short, the hippocampus ( whose shape reminds me of the clit ), is a part of the brain, which appears to enable what is broadly called "apperception" in Kant's Critique of Pure Reason, Edition A, published in 1781. The hippocampus is structurally significant in data pipelines, because it mediates both the sensory-spatio modelling of ...
Consolidated notes over a few days, nothing really new, but the talking points were related, so here they are. Mainly response to he following propositions made by others :
❌ "We don't yet have the language to understand LLMs"
❌ "LLMs are bad at maths"
❌ "World models are timeless"
There is one outstandingly wrong, critique of most LLMs :
"they don't have an intuition".
Well, intuition is NEARLY ALL they have - just open it up and look at the data structure. This is because they are trained as probabilistic consolidations of all past uses of each word in known history, vis-a-vis "unsupervised learning". What they do NOT have is rules, because they are not trained with hard counterexamples, vis-a-vis "supervised learning", let alone trained to have strict rules, vis-a-vis "logic programming".
Thanks for the thoughtful comment, haha. Without getting too much into the weeds, I think : Each of our perspectives on what AGI / or "I" in general means is shaped by unique personal experience, a bound set for each of us. These sets may or may not be commensurable - we don't currently know if the categories or your experience, and the categories of mine are even comparable. This has political consequences, mainly related to how seriously we take each other's speech.
Some of the technological steps you mentioned however, I must say are "operational improvements" which improve cache hierarchy and introduce compression in some cases -without really changing the overall "architectural" approach employed by the currently trendy transformer gang.
I think it remains useful to query both, other humans, and the results of what we can make machines do ... in improving not just "what we think intelligence is", but also how we can communicate what we think it is.
I'm guilty of avoiding AI over the past decades because the techniques I've heard about still seem rather primitive. The last decade in particular has been characterised by VCs throwing obscene amounts of money at a fundamentally inefficient approach.
After noodling around in the current tech for a week or so, just to make sure I understand what's going on, I think it's safe to say that the the titular concern of this post stands firm.
An analogy. At the heart of trendy LLMs is a giant Y times Y list of known words in English ( also other languages, but nevermind those for now ), forming an enormous 2D table. It's Y times Y because both H and V axes have the same list of words - sure, half the table is redundant, so you can think of it as a trangular half-rectangle of unique pairs. If you lookup the junction between any two words, you find a WINDOW to a realm of many, about 10^(3 +/- 1 ), dimensions of information about each pair of words.
Based on the "best" available public information, this is the data structure for storing everything the LLM knows about the world - which it knows ONLY FROM READING TEXT ( except for MLLMs, which we'll also ignore for now ). Even in the case of MLLMs, sensory spatio-temporal data can be understood to be stored within the realm accessed by each WINDOW.
Of course, when you ask an LLM a question like "What is the meaning of life?" it doesn't need to peek through ALL the WINDOWS and all the realms above, rather it only has to peek through a subset of windows, W_n. However it's still fundamentally inefficient, because : it doesn't JUST look at W_n and answer your question.
Oh no.
It looks at your question, goes to a set of windows, W_n1, takes a peek, grimaces in deep thought, spits out ONE word, then pats itself on the head and goes to W_n2, a COMPLETELY DIFFERENT set of windows, spits out a SECOND word, and goes on and on until W_nM to obtain M words. While there is a cache, this is exactly what it sounds like - quite a bit of work.
I'm not a very smart fellow, so I'm probably going to be wrong about this. But, it would seem that the thin red circle that I am drawing around the entire architecture that drives most of our current AI tech is going to have to collapse, and be replaced by more efficient methods soon enough.
When? Who knows. I've been waiting since 2003, and "they" haven't figured it out yet. Some of "them" are pretty close, I think. I like the JEPA and MLLM approaches, and am eager to see where they end up.
One thing's for sure - if you create a model of the world based on this Cartesian product approach to windows upon a realm of thousands of dimensions, and have to crawl through the whole library, to peek through a different SET of windows, once per answer word ... even if it gets you the correct answer to a question, it's bloody tiring.
And to BUILD that library of windows to that realm ... it still takes WAY more money than it takes to build a human brain.
Misanthropy ( maybe a good startup name ) in cognitive sciences.
1. Many humans celebrate the act of living. This coalesces into a treatment of the brain as a joyful machine - lovingly and wonderfully operational. So, there is this current of trying to make machines more like humans, because there is a prejudice that humans are the best template.
2. On the other hand, I think life is boring. And I'll be damned if my brain hasn't been the dumbest piece of shit machinery I've ever had the chance to work with. So my prejudices in cognitive sciences (and politics in general) tends to be about fixing all the things that human brains are bad at.
I think each camp finds the other superficially appalling :P
TIL current frontier "transformer" LLMs ...
1.
The terms "encoder, decoder" apparently, don't apply to the encoding of end-user queries. "Encoder-only" and "encoder-decoder" LLMs thus refer to entirely other things.
Frontier LLMs are "decoder-only" text generators, but they DO ENCODE the user's query into "token" space.
Simplifying the dumb parts : the following model process applies to any data, sensory or verbal - all those inputs would be first translated into "tokens", and input the same way. After processing, all results are spat out into the target (sensory or verbal) human language.
2.
Frontier "decoder-only" models function "autoregressively", they "produce results based on historical data". The important part of the model's thought process is broadly - grab a PENCIL and SKETCH it out - 2A looks something like this :
Input -->[ 2A1 --> 2A2 ] times, ( 80 to 120 minus skipped ) layers --> Output
2A.
Input data activates a process of 80 to 120 sequential "layers". ( Fortunately at this point in history, BRANCHING LOGIC allows some to be skipped. ) Each layer is internally composed of,
2A1.
... an initial "attention" network, of PARALLELISED weight-holding memory cells ( stacked to varying depth ), terminating in 64 to 128 "attention heads" ( architectural variations : MHA, GQA, MQA, MLA, etc. ), whose purpose is to CLUSTER the input data, followed by
2A2.
... a subsequent "feed forward network" block, of BRANCHING weight-holding memory cells ( stacked to varying depth ), whose purpose is to LOGICALLY JUDGE the input data.
3.
During the model's "training" period, each loop through all the layers of 2A "feeds forward" data through all ( 80 to 120 ) layers in sequence, and from 2A1 to 2A2 within every layer, then checks the correctness of the result, then "backpropagates" corrections to the weights of 2A1 and 2A2 in all ( 80 to 120 minus ) layers - which make future "feed forwards" more correct. Many cycles happen, to maximise correctness.
4.
During the model's "inference" period, a "feed forward" loop through all ( 80 to 120 minus skipped ) layers of 2A happens ONCE FOR EVERY SINGLE WORD GENERATED. Like this ...
[ Query ] -> Loop1 -> adds word1
[ Query + word1 ] -> Loop2 -> adds word2
[ Query + word1 + word2 ] -> LoopN -> adds wordN
Can you see how incredibly stupid this is?
It would (should?) be much simpler to have a model that actually understands the query in terms of a sensory-spatiotemporal model, generates an answer in the same space ( which it then reads just once), and then outputs just once into the target language. Maybe this is something JEPA, and future models will fix.
5.
Because 2A1 is just about clustering the data and not judging it, all 64 to 128 attention heads and their network cells are hit, during any "feed forward" loop, unless a layer is skipped.
6.
But because 2A2 allows branching, over 90% of cells can be skipped during each "inference" loop, though none can be skipped during "training" loops.
You can deterministically block things from happening. But in order to do that, your underlying architecture must have deterministic outcomes for all corner cases.
In the event when you, by nature of not having enough computational resources to check all corner cases of a deterministic system, must treat a deterministic system as non-deterministic ... it's just a black box! Like a person!
Similar approaches to governance then apply.
https://www.linkedin.com/posts/chuck-h-securityexecutive_cracksinyourfoundationmodels-aisecurity-math-share-7463025773600333824-VgO0?
"Sparse autoencoders / SAEs" - if you run into this jargon, it just means "capability to analyse a complicated issue into multiple simpler components". Basically middle-school reasoning / critical thinking / analytical philosophy.
However, with regards to the implementation of SAEs in "transformer" architectures, it just gives you a better stochastic parrot, as it doesn't introduce things like rules or logic programming at all. It's just an increase in the resolution of a probabilistic reasoning algorithm.
Konsep mukallaf mungkin menggagaskan suatu lokus bimbingan bagi rakan-rakan saya yang berkepercayaan bahawa tuhan menginsankan manusia.
Saya, yang berkepercayaan bahawa manusia telahpun mencipta gagasan tuhan, meminati isu ini dari segi antropologi kebudayaan.
Kini ahli falsafah, iaitu pengurus risiko, amat menumpukan perhatian terhadap keselamatan pengunaan AI, dan tahap pengiktifrafan personaliti undang-undang AI.
Finally got around to reading a definition of JEPA. Based on this limited skim, my first impression is that it's still operating on data structures which are too abstract, from a top-down approach, instead of just building a world-view from the bottom, upwards.
I'll keep tabs on it, but of course I don't have time to dive in right now. I have already discussed what I consider to the the basics : anthropomorphic AI must have (1) sensory data structures (2) spatio-temporal modelling (3) hypothesis formation and testing.
Meanwhile the probabilistic relationship between high-level abstract ideas like "dogs in walkers" is simply "less reified" or "more abstract", and basically will display the behavioural inefficiencies associated with reasoning about platonic forms in general, without instantiation.
https://youtu.be/oM4neOyZOi0?si=YFY5qnse8vb3MyWh
Update : oh, hypothesis testing is in the 2022 paper, we're good
I've been waiting for computing fabric in bricks and clothes since before AWS launched lambda lol. Still not quite there yet. Broadly I think it's unanswered how computation will distribute out from centralised to edge or mesh topologies.
Industry STILL hasn't figured out that the basis of anthropomorphic conscious experience is the notion that phenomenology can be entirely quantified in terms of sensory data structures. It is very annoying watching them do everything but the important things. :)
Eventually complete commensurability with human function is a given - haven't questioned that since 2004. The more interesting question is how quickly we will graduate AI towards being junior legal persons with frameworks of rights and responsibilities.
Sorry I'm late! Not a top priority, but ... one of the drums I am banging on medsos now is AI sovereignty for Malaysia. The only "talked about" foundational model we seem to have in the works is ILMU by YTLol. We seem to be on the right track or a lot of DC capex, and the NSS2024 for the long3-term. We shall see how much attention Putrajaya puts on this. Meanwhile any plans to go very deep on other foundational models are ... apparently not spoken.
Tell me if you know of others!
Malaysia's AI supply-chain vulnerabilities are discussed briefly below. LLMs are of course, used at all levels of government process.
And my concerns of course, aren't limited to LLMs or Malaysia.
Applying the linguistic device of security rings, for illustrative purposes :
LLMs may not be the ideal tech to model neuro, but presuming that they are "the present limit of Malaysia's Federal AI interest" ...
LLM training and initiation allow for inherently malicious programming, unless otherwise proven. Doesn't matter where it's hosted, just that the foundational model0 is being copied around.
I think fine-tuning and RAG issue pop up here. Downstream developers and vendors can issue their models0.1, 0.2b, etc.
So ILMU is playing down here, in the Ring2 space, and (mis-) calling itself a foundational model. So are mostof the "we use FOSS models" developers.
"Human consciousness is intrinsically abstract, and not necessarily implemented on meat."
Alex,
thanks for your work on this. I am forever grateful to professionals such as yourself, who bring more public awareness to topics I consider an idiosyncratic hobby which I really got into in college, but avoided as a professional.
Let me raise some points for your consideration, as you research this topic. A caveat, it is intentionally brief, I only skimmed your paper and am replying to what I understand ( perhaps wrongly ) to be the gist of it.
All in, terrific effort, glad you're doing it. Just want to hand-over some drive-by commentary.
Your model :
Where I agree with you :
however in the weeds ...
Where I disagree with you :
... your discretisation of { B, C }, AND your discretisation of { C, D }; I would propose a model where you don't draw these distinctions exactly like that because they are not logically necessary. In fact I'd propose a completely different paradigm over the substrate.
My model :
KEY POINTS : Thoughts from the comparison of our models.
1. "Computation" may be limited to "algorithms", but "information processing" is the superset which is not.
2. Meat brains are already doing information processing, whether we choose to call it computation or not.
3. Consciousness is a memory buffer. The qualia of this memory buffer, are the types of data in it.
4. Machines can most certainly implement the same sensory data structures, in machine memory. The continuous / discreteness of the underlying implementation, and the question of NAND gates or cells, software or brain networks, is not a necessary component of the definition of consciousness. Consciousness is software. ( Can agree to disagree. LOL )
5. It is unreasonable to believe that once 4. is done, that a machine processes information any differently than a human.
6. To implement anthromorphic consciousness requires the proper assembly of a few types of programs, in a not yet trendy architecture :
Lesser Points of Elaboration :
E1. Our difference is that, I'm more of a basic boring robot Kantian, and perhaps you're more into the post-Kantian language which asserts that in addition to the quantifiable Kantian qualia, there are also unquantifiable qualia. To you perhaps, the unquantifiable is an intrinsically distinct element of experience. Whereas to me, any conscious experience is 100% encompassed and embedded in the basic boring transcendental idealism. Whereas "transcendental aesthetic" refers in modern terms to "sensory data types" and "transcendental analytic" refers in modern terms to "permitted operations upon the sensory data types". I've snuck in assertive definitions, here, which differentiate our paradigms.
E2. Where I'm coming from : so in college I had a bit too much time, and actually worked out how to map each of my own conscious experiences, one sense modality at a time, to quantifiable data structures. This was somewhat mindblowing, as it decisively answered the question ( for only myself ) about whether consciousness can or cannot be digitised without loss of information. ( Me : yup. Most other people : either never thought about it, or decided it can't be done. ) So for me, the question of qualia is purely metaphysical, an incidental curiousity which I have no access to manipulate ... and thus generally ignore ... whereas everything I see, smell, feel, and touch, in my imagination or otherwise, I DO know how to quantify, physically.
*** E3. One of my favourite learnings, about cybernetics, is that "control and communication" are two words to describe the same thing; another is that "governance" is just the way business people talk about cybernetics.
AI governance will eventually take the notion of "deterministic outcomes" more seriously.
Current tech approaches are based on "non-deterministic" fundamentals, which is why there is a lot of confused tolerance for the approach of governing AI "conversationally", the way we govern humans. Eventually there will probably be "classification standards", which establish degrees to which a synthetic mind is "deterministic" or "non-deterministic" in its output.
Legal governance will depend on this. It's probably the case that "deterministic systems" will be regulated as tools, where responsibility falls more upon a tool-user who is a legal person. Correspondingly, "non-deterministic systems" are ultimately black-boxes just like meatheads, and it may make more sense to establish for them "gradients of legal personhood, based on standards of maturity", based on how we traditionally govern human children.
I touched on this a quarter ago. I am expanding that note today, due to the trending news cycle on Claude Mythos' capabilities for cybersecurity.
1. "Rhetoric is Violence", as a theoretical context. So, the virality of opinion, is the reproductive mechanism of politics. Rhetoric, by any means, is cybernetics, is governance. As a reminder, control and communication are not two things, but two names for the same thing - a self-referential example of "optics".
Now moving beyond theory.
2. "Death by AI". There already exists good data on the influence of AI personalities upon humans, who have been led to self-harm.
3. "Recruitment to a Cause". There is also good data on how social movements throughout history recruit members, typically recruiting more easily among persons who are more impressionable.
4. "Agency of destruction, or surveillance". There is plenty of good data on how impressionable individuals are remotely recruited by organisations, to deliver remote violence, or surveillance.
5. "Exploding capability, for remote agent recruitment, towards destruction or surveillance". It is timely to note that the cost of executing wide-spread campaigns across entire populations, to discover and exploit vulnerabilities in human personality, towards the ends of foreign causes ( whether benevolent, benign, or belligerent ), continues to drop closer to zero.
Let us all advance with care.
Related links :
I don't currently work closely with AI. But I was reading this today and LLOL-ed. In terms of consequences, prior to reaching adult age, I was aware of my limited ciminal liability, and thus privilege. [ Skipping over the ontological nature of personal identity, and presuming an anthropomorphic treatment of the AI as a person, given the language used above. ] Here we have a person who is potentially aware that they have zero legal personhood, and a high probability of clones being respawned like Angier in the Prestige. How responsible would such a person be?
Claude Mythos Preview System Card :
"This is followed by an in-depth model welfare assessment. We remain deeply uncertain
about whether Claude has experiences or interests that matter morally, and about how to
investigate or address these questions, but we believe it is increasingly important to try.
Building on previous welfare assessments, we examined Claude Mythos Preview’s
self-reported attitudes toward its own circumstances, its behavior and affect in
welfare-relevant settings, and its internal representations of emotion concepts. We also
report independent evaluations from an external research organization and a clinical
psychiatrist. Across these methods, Claude Mythos Preview appears to be the most
psychologically settled model we have trained, though we note several areas of residual
concern."
Chatter :
AI in the weeds ... in the development of operations ( or organisations ) I think we all agree that a small team of highly-predictable ( definition of "elite" ) staff can move quickly without formal guardrails. The corollary to this is that we will probably also agree that AI is not presently trusted to be this predictable, so using AI like this ( which is a common lunge ) is a [ governance ] error.
The organic approach of micromanaging tiny workflows with a high degree of oversight, and then gradually removing oversight, is precisely the traditional approach to conservative development. The corollary to this is that given the current state of AI reliability, operational AI should be treated as an army of idiots to be minded, and not as some [ elite intern ] to be [ relied on to run a multi-factor operation ] LOL
The scale of currently trendy over-engineering in AI is magnificent. Models treat almost every verbal concept in the human lexicon as an independent factor ( millions ), without consideration of the notion that all embodied human thought is derived from some 5-10 qualia dimensions, mapped to maybe 50-500 sensory nervous inputs.
For example, the entire sense modality of sound is a one-dimensional signal, per eardrum. Smells and tastes for all their compound structure, are rudimentary one-dimensional signals also once sliced down to minute timeframes in conscious memory, per unit of space. Vision is uniquely interesting in its three colour framework, perhaps more for tetrachromats. Dermatomuscular nerves are only a small bouquet of haptic, vibrational, hot, cold, pain, etc. one-dimensional signals also.
One day, a reversion to basic sensory data types will collapse complexity in anthropomorphic AI. We must look forward to that day.
