I was shocked the first time I heard this phrase. It was from a guy I dated in 1976. He was from rural Kansas.

When I explained to him that it was offensive, he countered that it wasn't offensive because he didn't mean it to be offensive. "It's just what we say."

Same guy, 1978, addressing a large group of lesbians as "ladies" after they had made it clear that they were to be referred to as "women," not "ladies."

Once again, "It's not offensive if I didn't mean it to be offensive."

Some people are always going to be clueless. No way around it.