I fully understand your point.

I fully understand your point. What you describe is what I hate about story points as well. Having said this: the misuse of story points, estimates, plans, etc… is caused by the fact that the organization abuses them. Their mindset needs to change. If not, they will abuse anything.