| GPT-6 Astra, looped transformers, and hidden reasoning(magazine.sebastianraschka.com) | |
| 492 points by ModelForge 1 day ago | 158 comments | |
tl;dr: OpenAI's rumored GPT-6 Astra reportedly uses "looped transformers" (recurrent depth), an architectural technique where intermediate representations pass through the same transformer blocks multiple times, effectively increasing depth without adding parameters—an approach seen in models like Nanbeige, Ouro, and Universal Transformers. The author argues this looping likely isn't the reason Astra produces shorter/less monitorable reasoning traces; more capable models simply need less scratchpad, similar to how larger models in a family use fewer tokens than smaller siblings. Astra's biggest practical leap appears to be in computer-use tasks, likely trained via RL on OpenAI's reported fleet of Mac Minis. | |
HN Discussion:
| |