Linux

Anti-cheat that isn't kernel-level has been around for ages - and even with the kernel level anti-cheat people still manage to cheat. I don't think a special OS with it's own kernel, just for gaming, would be practical. Most people aren't going to install something like that, hell, most people that use a PC have no idea how installing an OS functions. Just have anti-cheat that isn't kernel level, and have a robust report/banning system. Team Fortress 2 was practically unplayable for a long time due to it's botting problems, but they managed to resolve it without kernel level anti-cheat.