Adrian has it right. Let me add a bit of emphasis and a little more direct insight ...
I played saxophone for over 15 years -- from its being my very first instrument in 6th grade until I stopped playing in graduate school. I was "classically" trained (in part by a Masters student at Eastman), but also trained to a lesser degree by a local jazz musician. I was good at it (competitions, juries, etc.), both as an individual and in groups.
Here's my primary answer to the original question: Tuning, intonation, and tone quality. (Uh ... what more do you want from an instrument?)
The saxophone is a VERY easy instrument to learn to play. It's even easier than baritone (yes, i know: hard to believe). You put the reed on, you blow into it, and you push buttons to change the pitch. Sounds come out, and they're more or less correct. You're up and running as a sax player in a couple of days. The embouchure is MUCH less demanding than that of the clarinet or flute (which have very similar button arrangements).
It's that "more or less" part that's the major problem. There's an old saxophone joke that the difference between a saxophone and a lawn mower is that you can tune a lawnmower. It's funny because it's true. How DO you affect the tuning/intonation of a saxophone? Two ways: (1) buy an expensive instrument that has excellent intonation, and/or (2) control it completely through your embouchure. You'll notice that when ensembles tune, the sax players will often pull out their mouthpieces a bit or push them in a bit. How much do you think that actually affects the pitch? Right. Now things like clarinets and flutes (not to mention the double reeds) have much the same problem, but (possibly because of the more dramatic conical bore?) in saxophones it's very pronounced.
Student level saxophones typically really suck in terms of pitch and intonation. Saxophone players in the schools (who are mostly taking lessons from a school "music teacher" who is almost never a saxophonist) are never taught how to play in tune uniformly, and their instruments are not helping them. Later in life, these people end up in community bands. They do not know how to play in tune, and they usually don't know what "in tune" means because they've never trained their ears to do it.
When I showed up for my first lesson (at about age 13 in the early 60s -- before some of the really good saxophones were manufactured) with my Eastman guy, I saw that he had masking tape all over his instrument. Why? "It's the only way to get it to play basically in tune."
The saxophone is a very easy instrument to learn to play. It is a VERY VERY difficult instrument to learn to play in tune and with good tone production. In the family, the baritone is generally the best of lot in terms of tone quality, pitch, intonation, and tone quality. I don't know why this is.
In the past year I've gone from one community band where the saxophone section was dreadful to another where the section is actually good. What's the difference? Better (more expensive) instruments and better training and musicianship. But that's quite unusual for a saxophone section in community band. Quite unusual. In general, amateur saxophone players who show up in community bands have significantly poorer equipment and are just not as good musicians as most of the rest of the players. It's their culture.
Ladies, don't let your children grow up to be saxophonists -- unless you have them trained for the real rodeo just like other musicians.
Gary Merrill
Wessex EEb Bass tuba (DW 3XL or 2XL)
Mack Brass Compensating Euph (DE N106, Euph J, J9 euph)
Amati Oval Euph (DE 104, Euph J, J6 euph)
1924 Buescher 3-valve Eb tuba (with std US receiver), Kelly 25
Schiller American Heritage 7B clone bass trombone (DE LB K/K10/112/14 Lexan, Brass Ark MV50R)
1947 Olds "Standard" trombone (Olds #3)