Socialists don't hate markets, they hate workers not having any power or democratic choice in how they interact in the market.
Workers owning the means of production just means the workers are doing the same work but they are in ownership of the factory and the profits. They will still sell the products they produce in a marketplace.
All types of governance and economic systems are susceptible to despotism.
It takes a constantly educated and involved population to fight it.