We can debate exactly where to draw the line, for sure. And, honestly, it should be calculated as a multiple of minimum wage or 'poverty level' or something, to avoid the number becoming obsolete due to inflation.*

But, really, anything between $10M and $200M would be a fairly reasonable place to draw the line. The important part is to tackle the ultra-rich, the ones for whom even hundreds of millions is chump change. (Personally, I'm letting it edge up a little higher to account for small(ish) business owners and people/farmers who might own a particularly valuable plot or two of land.)

*Edit: $50M is ~6,896,551x the federal minimum wage of $7.25. For simplicity's sake, let's round the limit up to 7,000,000x. Which would make the limit $50.75M in total assets. If the rich fuckers want to be able to own more than that, they'll need to raise the minimum wage, which would then raise the limit accordingly. Raising the minimum wage to $10/hr would increase their limit to $70M, for example.