Mattis's confirmation, the first action taken by senators after Trump was sworn in, passed by a 98-1 majority.Lawmakers had first passed a special waiver allowing Mattis, who retired in 2013, to serve before a customary seven-year limit on former military personnel assuming the Pentagon's top post.
That waiver was the first law Trump signed as president.
The Senate also confirmed John Kelly, another retired Marine general, to head the Department of Homeland Security.
"These uniquely qualified leaders will immediately begin the important work of rebuilding our military, defending our nation and securing our borders," Trump said in a statement.
"I am proud to have these two American heroes join my administration."
Mattis has won accolades from both parties and many in the armed forces, and his path to the cabinet was relatively uncontroversial compared to that of many of Trump's other nominees.
A cornerstone of US democracy is that civilians, not people in uniform, control the military, and the commander-in-chief is the president.
Some in Congress initially raised eyebrows because Mattis, a 66-year-old Washington state native, had only retired from active duty in 2013.
Such a waiver was only granted once before, for the famous World War II General George Marshall who served under President Harry Truman from 1950-1951.
Mattis is known as a colorful commander and is famed for his pugnacious aphorisms. The media dubbed him "Mad Dog" for his battle-hardened swagger and the sort of blunt language Marines are famous for.
He has been quoted as saying, "Be polite, be professional, but have a plan to kill everyone you meet."
But Mattis also has a well-known cerebral side: he issued reading lists to Marines under his command, and instructed them that the most important territory on a battlefield is the space "between your ears."
While Mattis sailed through his confirmation, Democratic lawmakers are putting up stiff resistance to other Trump cabinet picks, including the proposed education secretary and the health and human services secretary nominee.
Mattis replaces Ashton Carter, a longtime Pentagon bureaucratic warrior who served as former president Barack Obama's fourth defense secretary.
