Mathematicians suck at writing. This is part of the reasons why they went into math in the first place, because they suck at writing. Sucking at writing doesn’t mean that mathematicians don’t write books though; in fact there are tons of math books written by mathematicians.
The problem is most of them suck.
Half of them are for geniuses written by geniuses.
The other half are for geniuses written by borderline geniuses.
By far my two favorite books are William’s Probability with Martingales and Trefethen’s NLA book. Both British authors. Both written in a highly colloquial style.
It’s really too bad academics have this ego-stroking urge to write to the highest denominator rather than to, say grad students or undergrad.