Israeli Prime Minister Benjamin Netanyahu and other officials gave Modi a red carpet welcome at Ben Gurion airport near Tel Aviv as he began his three-day visit.
The two leaders embraced heartily and gave brief speeches, pledging to work together on issues including security, agriculture and energy.
"It is my singular honour to be the first ever prime minister of India to undertake this groundbreaking visit to Israel," Modi said.
Netanyahu said "we've been waiting for you a long time. We've been waiting almost 70 years in fact, because yours is truly a historic visit".
Modi later visited the Yad Vashem Holocaust memorial in Jerusalem and was to have dinner with Netanyahu.
The two leaders are to hold talks on Wednesday.
Israel, constantly searching for allies at the United Nations and for new business partners, has portrayed the visit as a major diplomatic victory.
Israeli analysts have noted that Modi is not scheduled to travel to Ramallah to meet Palestinian leaders during his three-day stay, as is common for visiting dignitaries.
Modi did however meet Palestinian president Mahmud Abbas in New Delhi in May.
India is the world's biggest importer of defence equipment, and Israel has become one of its major suppliers.
Israeli media have reported that the two countries sign defence deals averaging more than $1 billion (880 million euros) a year.
In April, state-owned Israel Aerospace Industries said India would buy nearly $2 billion worth of weapons technology, making it the military exporting giant's largest ever defence contract.
The deal will see IAI provide India with an advanced defence system of medium-range surface-to-air missiles, launchers and communications technology.
'Make with India'
Modi and Netanyahu also hope to build on cooperation in areas including water and agriculture technology, sectors where Israel has excelled.
India has traditionally voiced its support for Palestinian statehood, long shunning ties with Israel.
But the frost has thawed in recent years, with India seeking closer defence ties -- particularly as it moves away from relying on traditional ally Russia for its military hardware.
Modi's visit marks 25 years since the two countries established diplomatic relations.
The Hindu nationalist makes the trip after his visit last week to Washington, where he and US President Donald Trump embraced each other as friends, vowing to work more closely on combatting terrorism, the war in Afghanistan and defence cooperation.
While ties with some US allies have been strained by Trump's complaints that Washington has been the loser in trade agreements, Modi appeared sensitive to his host's emphasis on transactional diplomacy.
Modi may find similar common ground with Netanyahu, Israel's longtime right-wing prime minister who hosted Trump in May.
India is currently the world's fastest growing major economy, a status that Modi is hoping to cement by drawing in more foreign investment.
He has sought to do that in part by encouraging manufacturers to do business in Asia's third-largest economy.
Israel's defence deals have included components that will be assembled in India.
"On this visit, your 'make in India' initiative meets my 'make with India' policy," Netanyahu said at the airport ceremony.
