Mainly, because I'm too lazy to take it off. :)
Actually, it's two reasons. One, there's the remote outside chance that it will deflect a rock (it's deflected at least two that I know of that left divots, but missed the windshield), and I don't want to risk stripping the screw holes taking it...