Sometimes; when you hear somebodies political take, you can instantly tell they fall in one of the two extremely simplistic, easily marketable positions. When this happens, the easiest thing to do is politely decline to discuss politics with them, because you know they lack the nesessary context to understand your well thought out, nuanced position.
If you never encounter this problem, I suspect you may be in one of these two camps, and would strongly encourage you to think about your politics in detail. Do you know why you believe what you do?
I'm assuming "I don't do politics" in this scenario is abstaining from debate, not abstaining from voting at all. Of course you go with the side that aligns best with your beliefs.
My point is, if you're talking to someone, and all they're doing is waiting to see which shibboleths you spit out to decide if they hate you or love you, then you aren't going to get any meaningful discussion out of them.