Not so much about affording proper attire, its more that clothing is shallow. I've interviewed people that plain over dress for the interview who turn out to be unqualified and nothing but snake oil salesmen and I've interviewed a person who wore a t-shirt and jeans and was by far the most qualified candidate.
My thinking is you are not getting paid for the interview. The interview is to see whether or not the person and the job is a good fit. To best help in determining that, I want the candidate relaxed. So I tell people wear whatever they feel comfortable in. There are limits of course, still gotta dress like you are in public lol

)
Comment