Significant proportion of people don't fit into binary sex definition, however much you cut and define it, people have all sorts of crazy combinations. And if your strict binary definition doesn't include all the subjects, it's not strict, not binary, and signifies not a lot.
"The observable universe is either Hydrogen or Helium, we don't let small percent of exceptions destroy this nice model we have"

Sure, the chromosomes and gametes exist, and are supported by biology... but how and why are we using those to define sex? Pinning down a precise definition of sex is incredibly difficult for biologists, the best definitions we have are a vague cloud of all of these vaguely related things. The only common thread is that all of these things differ between different humans, and have some (often tangiential) relation to fucking.

For any rigorous definition we come up with, that actually produces a distinct number of sexes, we can throw some curveballs at it that require the writing of exceptions and special clauses and caveats. We end up oversimplifying dozens of different variables into these broad categories that then lead to worse outcomes for the people who deviate from the dogmatic oversimplifications. And if we actually count each intersex condition and permutation and stage of transsexual transition as its own sex, then we end up with thousands of different sexes, and next to no predictive utility! It's a nightmare!

My philosophy is: If you can't create a model that is accurate, fair, and useful... maybe you're trying to create the wrong kind of model. Go back to the drawing board and reevaluate your foundational assumptions. Come back with a different core idea.

What we have ended up doing with the idea of biological sex would be absurd in any other scientific field. Imagine if chemists said "There are two elements: Hydrogen and helium. Together they make up 99% of all atoms, and everything else is a defect." That's what we're doing with sex.

All four of them were/are quite good musicians. Yes, even Ringo

Hmmm... Blackbird is better than their average, but it's not quite good.

The problem with the Beatles is that their beats suck ass. Their songs feel lethargic, meandering, aimless, and depressing because they have terrible beats. Blackbird actually has a beat, which is more than I can say for most of their music. But it doesn't have a good beat.

UNBELIEVABLE! YOU GUYS FORGOT THE MOST IMPORTANT PART OF WRITING A CATCHY SONG! It's gotta have a beat.

Objects exist

I just think the entire idea of distinct supermolecular objects is an arbitrary human convenience, rather than a scientific construct. Science doesn't have an answer to philosophical problems like the Ship of Theseus problem. I think that's because objects are not a scientific concept. Objects are a vibe.

Your fault for using object-oriented programming, also known as OOP.

You should try God-oriented Programming, or GOP for short.

It exists, but like, in the same way species exists. The variety and frequency of intersex conditions indicates that it's two clusters of traits that most species that show this characteristic fall into one or the other of and certain components are remarkably mutable.

That understanding is just fundamentally different from the traditional understanding of two entirely distinct bins that anyone who doesn't fall into one or the other is a strong outlier and the only alterations are castration or divine intervention (at least in euro-christian tradition).

The two bin model is going to work ok on most humans, it's not like it's an obviously wrong one. But as society and science have advanced we've found more intersex people that either didn't know (my cousin would have just been understood as barren, and nobody would've noticed my ex's mom having XY sex chromosomes) or didn't understand that it wasn't just some weird quirk that you either hide in shame or just don't feel is worth mentioning. And anywhere you try to draw a firm line or any trait you try to point to try to ignore the gray is going to leave you with an awkward grouping in some way