I think the haters only want it for that reason.
You are exactly correct and when you are the best player in the game, you are generally going to have the most haters. That's just how it works. As a Lakers fan, I've seen this several times. It's also why I think Lebron (and really all celebrities) are better served by saying as little as possible. Even if what they say makes sense and is true, people are going to twist it.