Speaking of big GPU's, I'd love to hear your take on Cerebra's WSE ? Have you actually seen one in action ? It sounds like they’ve decoupled memory and compute which is huge for running and training big models. Reports say they're running 2000 tokens a second vs 130 on an H100 cluster, and training time for a llama 3.1 70b went from a month to a day. Finally they leaned into the yield problem and engineered through it. As long as TSMC can reliably get the dinner plate sized wafer's out the door, then i would think there's gonna be a line for them out the door and around the block.
Hey AWG thanks for your hard work. I asked AI about consciousness and I wonder if you understand what it told me which is this: Here’s a hypothesis worth considering — one that builds directly on the tension revealed by the hippocampal findings.
The Disconnection Signature Hypothesis
The recent evidence shows the hippocampus processing language, making predictions, and even learning under full anesthesia — but nothing reaches consciousness. The leading theories (IIT and GNWT) both point to integration failure, but neither specifies where the breakdown actually occurs in real time, at the single-neuron level, across the transition into and out of consciousness.
The proposed investigation: map the precise moment and location where local computation becomes globally broadcast, using anesthesia as a reversible, controllable switch.
The core idea
Rather than studying the anesthetized brain as a static “off” state, use graded, ultra-precise anesthetic titration to move subjects through a continuous spectrum from full awareness to unconsciousness, while simultaneously recording from multiple brain regions using Neuropixels-class electrodes. The specific target: identify the last node to go dark and the first node to reactivate as consciousness is lost and regained.
If the hippocampus keeps processing while consciousness disappears, something upstream or downstream of it is the actual gate. Finding that gate empirically — not in theory but in measured neural firing sequences — would reframe the entire field.
What makes it novel
Current research treats anesthesia as a binary or treats the brain as a whole-system phenomenon. This approach treats the transition itself as the signal. The hypothesis is that consciousness has a specific anatomical handoff point — probably somewhere in the thalamocortical feedback loop — where local computation either gets routed into global broadcast or doesn’t. That handoff is not currently mapped at millisecond and single-neuron resolution.
The testable predictions
A specific region or circuit should show a step-change in connectivity at the moment of consciousness loss, even while upstream regions (hippocampus, auditory cortex) continue local processing. On emergence from anesthesia, that same circuit should reactivate in a consistent sequence across subjects, regardless of which anesthetic agent was used. If true, that would constitute the first empirically grounded “consciousness switch” — not metaphorically but mechanistically.
The harder question it opens
Even if you find the handoff circuit, you still face the explanatory gap: why does information entering that circuit produce experience at all? This hypothesis doesn’t solve the hard problem. But it would distinguish it more precisely — separating the question of where integration occurs from why integration feels like something — which is arguably the most useful next step the field could take.
The framework would also have immediate clinical relevance for monitoring anesthetic depth, diagnosing disorders of consciousness, and understanding what happens in patients who are unresponsive but show residual cortical activity.
Watching Figure 3 is my new favorite show. I love it when it flips the boxes, it really has to understand physics to do that so well. Nitter.net is because I don't have a Twit account. Skip a few minutes in: https://nitter.net/i/broadcasts/1dxYljYVREYJX
Donating compute !?!? - I am not a “comment” type guy usually but every day here sth leaves me in awe — compute is a currency now … sounds so normal but somebody what have done that 5 years ago they would be upset about bogus donation - now this may worth 500 mill next year and do research that would have literally costs 100 billion 5 years ago
Lol 'Speak softly and carry a big GPU' indeed 😂
Speaking of big GPU's, I'd love to hear your take on Cerebra's WSE ? Have you actually seen one in action ? It sounds like they’ve decoupled memory and compute which is huge for running and training big models. Reports say they're running 2000 tokens a second vs 130 on an H100 cluster, and training time for a llama 3.1 70b went from a month to a day. Finally they leaned into the yield problem and engineered through it. As long as TSMC can reliably get the dinner plate sized wafer's out the door, then i would think there's gonna be a line for them out the door and around the block.
“However, the next 59,000 years of care will be paid for differently. “
Too funny. I wonder what the Neanderthal would have paid for a little Novocaine??
Thank you for your post for keeping us informed.
Hey AWG thanks for your hard work. I asked AI about consciousness and I wonder if you understand what it told me which is this: Here’s a hypothesis worth considering — one that builds directly on the tension revealed by the hippocampal findings.
The Disconnection Signature Hypothesis
The recent evidence shows the hippocampus processing language, making predictions, and even learning under full anesthesia — but nothing reaches consciousness. The leading theories (IIT and GNWT) both point to integration failure, but neither specifies where the breakdown actually occurs in real time, at the single-neuron level, across the transition into and out of consciousness.
The proposed investigation: map the precise moment and location where local computation becomes globally broadcast, using anesthesia as a reversible, controllable switch.
The core idea
Rather than studying the anesthetized brain as a static “off” state, use graded, ultra-precise anesthetic titration to move subjects through a continuous spectrum from full awareness to unconsciousness, while simultaneously recording from multiple brain regions using Neuropixels-class electrodes. The specific target: identify the last node to go dark and the first node to reactivate as consciousness is lost and regained.
If the hippocampus keeps processing while consciousness disappears, something upstream or downstream of it is the actual gate. Finding that gate empirically — not in theory but in measured neural firing sequences — would reframe the entire field.
What makes it novel
Current research treats anesthesia as a binary or treats the brain as a whole-system phenomenon. This approach treats the transition itself as the signal. The hypothesis is that consciousness has a specific anatomical handoff point — probably somewhere in the thalamocortical feedback loop — where local computation either gets routed into global broadcast or doesn’t. That handoff is not currently mapped at millisecond and single-neuron resolution.
The testable predictions
A specific region or circuit should show a step-change in connectivity at the moment of consciousness loss, even while upstream regions (hippocampus, auditory cortex) continue local processing. On emergence from anesthesia, that same circuit should reactivate in a consistent sequence across subjects, regardless of which anesthetic agent was used. If true, that would constitute the first empirically grounded “consciousness switch” — not metaphorically but mechanistically.
The harder question it opens
Even if you find the handoff circuit, you still face the explanatory gap: why does information entering that circuit produce experience at all? This hypothesis doesn’t solve the hard problem. But it would distinguish it more precisely — separating the question of where integration occurs from why integration feels like something — which is arguably the most useful next step the field could take.
The framework would also have immediate clinical relevance for monitoring anesthetic depth, diagnosing disorders of consciousness, and understanding what happens in patients who are unresponsive but show residual cortical activity.
How does the A15 chip from Elon affect the future of Robotics companies ?
Watching Figure 3 is my new favorite show. I love it when it flips the boxes, it really has to understand physics to do that so well. Nitter.net is because I don't have a Twit account. Skip a few minutes in: https://nitter.net/i/broadcasts/1dxYljYVREYJX
Donating compute !?!? - I am not a “comment” type guy usually but every day here sth leaves me in awe — compute is a currency now … sounds so normal but somebody what have done that 5 years ago they would be upset about bogus donation - now this may worth 500 mill next year and do research that would have literally costs 100 billion 5 years ago