My brother has a pretty severe mental disability. When he was younger, though, it seemed like he had autistic-like tendencies, and that's what he was diagnosed as. So I grew up calling him that. However, pretty much every time I've used that term I feel like people get a misconception about what he's actually like. Like literally almost every time.
Now I honestly just tell people he's mentally disabled or brain-damaged because they seem to understand what I mean a lot better. Whereas anyone who drops the word 'autism' immediately has to deal with all these people who have no idea what it is, or have their own conceptions of it. I almost don't even really know what it is anymore. It's so poorly defined and tossed around so carelessly that it's really hard to say what, exactly, autism is, or what people seem to think it is.
But yeah. I totally get what you mean.