By comparison, xAI's supercomputer capacity isn't public but is estimated to be around 500 exaflops.
Edit: And that's just fucking X of all companies so imagine what an actual serious company like Google or Microsoft could have cooked up.
So while this is impressive, going CPU-only has probably resulted in a computer which costs an order of magnitude more while performing an order of magnitude less.
There is a reason the GPU architecture is still king in this aspect.