You've made the statement "We're closer than we think to travelling to other stars", then you've contradicted it almost immediately by saying "We haven't achieved interstellar travel, and who's to say we ever will?" Exactly... We may be much much farther from interstellar travel than some of us think, or like to think.

I'd suggest that the Fermi Paradox seems like a paradox because we moderns are so impressed with our technological successes (e.g. rockets, nuclear power) that we think we'll soon able to go wherever we like and do whatever we want there. Which may be a serious illusion. And if it is an illusion, then the so-called paradox evaporates.