No, you're right. The further away something is, observing it is like looking back in time. Technically, even looking at someone waving at you from even a mile away is looking back in time (though that's 1/1000th of a second delay or something)
Quite a few stars that we see in the night sky are...