Dumoulin clocked 12 minutes and two seconds on the 9.7-kilometre course to don the maglia rosa.
Tour de France champion Chris Froome, who cut and grazed his right knee in a crash during the course reconnaissance earlier on Friday, was a distant 21st.
Another top contender, France's Thibaut Pinot, was 16th, 33 seconds off the pace.
"I had been feeling good lately and today I took time off of my rivals, that's all I wanted," said Dumoulin. "It was a very challenging, very technical course but for me it was okay."
Froome lost 37 seconds on the Team Sunweb rider.
"If you crash in the recon, it doesn't make you very confident for the real race so definitely he (Froome) lost a bit more time in his performance than he would normally do and it was very unfortunate for him," Dumoulin said at the post-stage news conference.
"I didn't expect to take so much time off him."
Thousands of residents lined the route to witness an event never before seen in the city. They cheered loudly as each rider whizzed past in Israeli West Jerusalem in hot, overcast conditions.
The second stage on Saturday will see the riders start in the northern port city of Haifa and ride 167-kilometres south along the coast to the finish in Tel Aviv.
The last of the 101st Giro's three stages in Israel will take the riders on Sunday from Beersheba through the Negev Desert to the Red Sea resort of Eilat, a ride of 229km.
(Reporting by Julien Pretot and Ori Lewis; Editing by Christian Radnedge)
