It seems to me that the United States has nothing to worry about. The Japanese take the rules of entry into the country for the Olympic Games very seriously during the pandemic. As far as I know, fans are not allowed to attend the games at all. Only athletes and journalists are allowed.
In general, Japan is not the best country for traveling right now.
It’s best to go to Europe now. It’s easy when you have a vaccine or a negative coronavirus test result. This summer, the whole family traveled around Germany by train. It was very cool and exciting.