Let me make this simple for you as you clearly don't seem to understand.
Let's look at these 3 words very carefully. 1. Not. 2. All 3. Men. Not all men.
Nowhere in these 3 words does it say anything about disregarding victims of sexual assault. Nowhere does it say anything about suggesting that the sexual assault of women isn't an important issue.
It's simply stating that it is not all men who commit these acts. That's it. Everything else you've written in this article is figment of your own imagination.
The phrase is actually very generous. It should be: Not Most Men. That would be far more accurate.
Words mean what they mean. They don't mean what you imagine them to mean.