The Kartarpur corridor leads from the border straight to the Gurdwara Darbar Sahib Kartarpur, 4km (2.5 miles) away.
But tensions between India and Pakistan have made the shrine hard to access in recent years.
However, they reached an agreement over access last month, fulfilling a long-standing wish of the Sikh community.
The corridor - which is in Punjab, a region divided during the partition of British India in 1947 - has opened just days before the 550th anniversary of the birth of Sikhism's founder, Guru Nanak, on 12 November.
There are only two major border crossings between the two countries, at Wagah between Amritsar in India and Lahore in Pakistan and further south at Munabao, where the train running between India's Jodhpur and Pakistan's Karachi stops.
What's happening?
The first Indian pilgrims to use the Kartarpur corridor included former Indian Prime Minister Manmohan Singh; Amarinder Singh, the chief minister of India's Punjab state; and 150 Indian parliamentarians.
AFP
Indian Prime Minister Narendra Modi and his Pakistani counterpart, Imran Khan, inaugurated the corridor on either side of the border on Saturday morning.
Devotees from all over the world, including the UK and Canada, are also visiting for the celebrations.
Officials say the corridor can accommodate up to 5,000 pilgrims per day, with up to 10,000 able to visit the shrine every day.
AFP
Indian visitors to the shrine will require a passport but they will be carrying travel permits instead of visas. However, they will not be allowed to leave the premises of the shrine in Pakistan or stay overnight.
What is the significance of the shrine?
The shrine was built to commemorate the site where Guru Nanak spent the last 18 years of his life. It is considered to be the second holiest site for Sikhism after Gurdwara Janam Asthan, also in Pakistan, which was built on the site where Guru Nanak was born.
The existing shrine in Kartarpur was built in 1925 after the original was destroyed by floods. It was restored by the Pakistani government in 2004.
However, the facilities have recently been upgraded to allow the complex to accommodate more pilgrims. Construction began soon after the corridor was officially announced in November last year.
Spread across 42 acres (17 hectares), the expansion includes a new courtyard, museum, library, dormitories, locker rooms, an immigration centre and an embankment to protect the shrine from floods.
The corridor also features a bridge which will allow visitors to cross over the Ravi river, which flows between the international border and the shrine.
Why is the corridor so significant?
It has been a longstanding wish of the Sikh community to open the corridor up, and it makes it much easier and cheaper to travel to the shrine from India.
AFP
"Our wish to go to Kartarpur is finally being fulfilled," Amandeep Kaur, who lives in the border village of Dera Baba Nanak on the Indian side, told BBC News Punjabi. "We will go and we shall bow our heads there. We have been separated from our Guru for a long time."
The village is just 7km from the Kartarpur gurdwara. But until now, Sikhs who wished to visit the shrine had to obtain a Pakistani visa and follow a circuitous route that often took them to Lahore city, more than 100km from Kartarpur.
Why did it take so long for India and Pakistan to strike a deal?
A historically tense relationship between India and Pakistan - who have fought three wars since 1947 - has made it difficult for a deal to be reached.
The two countries first discussed the corridor in 1998, and then again in 2004 and 2008, but these talks never led to anything concrete.
The opening of the corridor comes in the wake of soaring tensions between the nuclear-armed neighbours.
AFP
In February, India launched air strikes against militants in Pakistani territory in response to a suicide attack in Indian-administered Kashmir which killed 40 Indian soldiers. The Pakistan-based Jaish-e-Mohammad (JeM) group said it had carried out that attack.
In August, India stripped the part of Kashmir it administers of its partial autonomy, sparking sharp criticism from Pakistan. Since then, Pakistani Prime Minister Imran Khan has been lobbying for international support against the move, which he has described as "illegal".
But analysts have said while this is a significant development, it would be wrong to suggest that what lies ahead is a peace process as India and Pakistan are still at loggerheads.
AFP
Before the deal was signed, India had said it was "disappointed" with Islamabad for charging pilgrims a $20 (£15) service fee to cross the border - but then said it would go ahead and sign the agreement.
There was also some confusion over whether Indian visitors require a passport.
On 1 November, Pakistan's Prime Minister Imran Khan said any valid ID would suffice, but news reports on Wednesday claimed that Pakistan's army spokesman said all visitors would require a passport.