What does he think of me?

don't know how to help you really, I'd just say take it easy for now. But he might have meant his house is cold as in nasty, not as in chilly? Random thought... good luck