The International Olympic Committee has said the Games could still begin on 24 July despite the coronavirus pandemic causing other events to be cancelled.
Coe told BBC Sport: "Let's not make a precipitous decision when we don't have to four months out."
However, he added: "If you had to ease that date, you'd have to ease it. It is possible. Anything is possible."
Speaking on the Today Programme on BBC Radio 4, Coe, who was chair of the London 2012 organising committee, also said: "Events are changing by the hour but it is not a decision that has to be made at the moment.
"We're trying to manage the situation with the information we have but there is not a great deal of information.
"The temperature in the room with the IOC is, nobody is saying we're going to the Games come what may."
The IOC says it held "constructive" talks with athlete representatives about the coronavirus crisis earlier this week.
President Thomas Bach admitted he was "confronted with many questions" over qualification and restrictions but insisted that "everybody realised that we still have more than four months to go" until Tokyo 2020.
On Thursday, Tokyo 2020 organisers received the Olympic flame in a scaled-down ceremony in Athens' Panathenaic stadium, site of the first modern Games in Greece in 1896.
The flame was then transferred into a small holder to travel to Japan on a special aircraft named 'Tokyo 2020 Go.'
In a video message Tokyo 2020 organising chief Yoshiro Mori said he hoped the flame's arrival would help "shake off the dark clouds hanging over the world".
When asked about the prospect of pushing the Olympics back a year to 2021, Coe replied: "It seems on the surface an easy proposition but athletics has its worlds on that date, the Euros in football have been moved back a year.
"The sporting calendar is a complicated matrix and it is not easy to move from one year to the next. It would be ridiculous to say anything is ruled out at the moment. The whole world wants clarity; we're no different from any other sector.
"In athletics we've postponed three Diamond League meetings but not all, because we don't have to make that decision. Everybody accepts this is a fast-moving environment. I need to be careful I'm not removing the earning potential from athletes any more than I need to at this moment."
Not 'much of a choice' but to call them off
In contrast to Coe, former Olympic champion rower Sir Matthew Pinsent said he did not see "much of a choice" but to postpone or cancel the Games.
"I just think there are bigger things to worry about at this stage on a global front," Britain's four-time Olympic gold medallist told BBC Radio 4.
"For most of the European countries as well as Asian countries, organised sport in any meaningful way has ceased, and that's from government advice. I don't see there's any way forward for an Olympic athlete to train effectively - even as an individual, but particularly in a team environment.
"Obviously I came from rowing, and rowing squads around Europe and the world are essentially downing tools and going into lockdown as their individual governments are mandating that they should - I just think it's unfair.
"It's unfair for the Olympics to say we're going to carry on. There are the two big forces in an Olympic athlete's life, which is the Olympics and everything else. And those two things are pulling in different directions at this moment."
Asked why he felt the IOC was insisting the Games would go ahead, Pinsent added: "I think they feel a responsibility to Tokyo.
"We know having hosted in 2012 that seven-year build up is a crescendo of energy and concentration and effort on behalf of the city and on behalf of the nation and the government. Everybody takes a pride in it.
"I know that Tokyo have done exactly the same and actually the financial stakes are much higher for the host city than they are for the IOC."
Meanwhile British race walker Tom Bosworth has told BBC Sport that the Olympics should be put back until the autumn.
"It would be a very tough one but in my opinion I think it should be postponed to allow everybody to qualify, to allow the build-up to be correct for what should be a really great and really successful Olympic Games," he said.
"If I'm honest I don't think there is enough time to properly build towards a Games, whether that is build athlete profiles, build the teams, allow people to qualify who haven't qualified. I think for all involved a slight delay is probably the best option."