The Taiwanese man who was arrested at the scene said he was expressing his political views and had stolen the sword from a nearby history museum, police told AFP.
The presidential office in the centre of the capital Taipei is the headquarters of Taiwan's Beijing-sceptic President Tsai Ing-wen.
Relations with Chinese authorities have deteriorated since she took office last year as she has refused to agree to Beijing's stance that Taiwan is part of "one China".
The island is a self-ruling democracy, but Beijing still sees it as part of its territory to be reunited.
Political stance
The attacker "took a hammer and smashed a display case in a history museum to steal a samurai sword", a police official working on the incident, who did not want to be named, told AFP.
"A Chinese national flag was found in his backpack. He said he wanted to express his political stance by going to the presidential office," the official said.
The man, identified by police only by his family name Lu, attacked the officer as he tried to stop him entering the complex from a side gate, said presidential spokesman Alex Huang.
Lu, 51, is currently being questioned by police. He is unemployed and has no prior criminal record.
The injured guard is in a stable condition after being rushed to hospital for treatment to a wound to his neck, Huang said.
'Open house event'
Defence minister Feng Shih-kuan condemned the violence and praised the 24-year-old guard for bravely stopping the attacker.
The incident came as the presidential office hosted a family event for its staff, including their children.
"This was an open house event and I can't imagine what the outcome would have been if he were to get in with the sword," Feng told reporters.
TV footage showed Lu being carried away by four officers and put inside a police car at a side entrance to the presidential office, which has been cordoned off since the attack.
Target of attacks
Local media reported that he had repeatedly left pro-China messages in comment sections online, including praise for the Liaoning, China's only aircraft carrier.
The sword he used is carved with the words "Nanjing battle, 107 people killed", according to a photo released by police.
An employee at the Armed Forces Museum, from which Lu stole the sword, said it had been used by the Japanese military in the massacre of residents of the Chinese city of Nanjing in 1937.
The presidential office complex and its surroundings have been the target of attacks before.
In November 2014 a driver tried to smash his vehicle into the front door of the nearby presidential residence, saying he was protesting the health policies of Tsai's predecessor Ma Ying-jeou, who was in power at the time.
In January 2014 a man drove his truck through a bullet-proof screen and into the main gate of the presidential office, saying he was protesting over a judicial hearing involving his ex-wife.
There were no injuries from either incident, apart from one of the drivers who was hurt.
