I think you're right. Users do care. It's partly subconscious, but I also think most of them lack the vocabulary to express why they like a design, even in their heads. They think, "Wow, that looks great," and some part of their brain fires in excitement because they've never seen anything like that before. Just because they might not be able to express that feeling in words doesn't mean they don't care, though.