104
No datacenter needed now, just run it on your computer. Anything with 12gb+ of vram can run it comfortably at low quants. If they can come out with a good MOE model it will crush the competition. At this current rate of progress, China will out-compete foreign AI development within a year or two.
Honestly it's all up in the air at this point, I don't think anybody can predict what kind of AI will be ran on data centres and on consumer hardware in the next 5-10 years. Maybe it all deflates because we hit a wall with it, maybe your phone will run a model more powerful than today's top ones.
My guess is the really good models will keep running on data centres but will be prohibitively expensive and/or will have their use regulated by their government so only big institutions and companies get access to it and the users will be running dumber models on their own hardware.