Obama: “Well, I don’t know how many viewers you’re talking about that say that.”
Conners: “We do hear from some.”
Obama: “I hear from all kinds of viewers about everything.”
Conners: “I’m sure you do.”
You can hear Obama sighing as Conners asks the question. Now, to be fair, the reporter pulls the classic "some people say" or "some viewers do something," which is actually just a little broadcast trick to say I think this or I perceive this is what people think, so this is what people think. You see that when he starts to pull the "sometimes, maybe they think..." line.
But Obama has come under legitimate fire for his vacations. Conners, in a subsequent stand-up, criticizes him for both the time he has spent away and the money he has spent.