Opponents to plans to build a mosque in the regional Victorian city ran into counter-protests and a large police presence as they gathered for a rally on Saturday organised by the far-right United Patriots Front.
More than 400 police officers kept the opposing groups - estimated to number about 700 people - apart in central Bendigo amid concerns of violence.
Two of the four people arrested allegedly had knives, while a third had a flare, Superintendent Mick West told reporters after the rally.
One person was arrested for a minor assault.
Numbers at the rally were much lower than expected and apart from the arrests, police said the crowds were generally well-behaved.
The last major protest in Bendigo, on August 29, resulted in violent clashes between demonstrators from the far-right and left.
On Saturday, pro-diversity and anti-mosque protesters stood on either side of a police cordon.
Anti-mosque protesters shouted "Aussie, Aussie. Aussie, Oi Oi Oi" and "No mosque", while pro-diversity demonstrators yelled "Nazi scum off our streets".
United Patriots Front (UPF) spokesman Blair Cottrell - who is not from Bendigo - told supporters the rally reflected local concerns.
"Do you want a mosque in your community?" Mr Cottrell asked the crowd, who shouted their opposition.
He took aim at the government and local authorities for encouraging multiculturalism.
"Islam can only pose a threat to our nation if our weak leadership, or rather complete lack of leadership, is allowed to continue," he said.
One anti-mosque protester told AAP he was tired of being labelled a racist because he was worried about Muslim extremism.
"Let's say there is five per cent extremists in that group, in that mosque," Bendigo resident Alan Clark said.
"What is the council going to do to say sorry to us if something bad happens?"
Mr Clark believed Saturday's turnout was a positive one - "we're all peaceful, mate", he said.
Jerome Small, spokesman for the Campaign Against Racism and Fascism, said Mr Cottrell's "neo-Nazi" views show he is a "wannabe fuhrer".
UPF's political aims were disturbing, Mr Small told the crowd.
"They are hardened fascists and what they are coming to Bendigo to do is to try and build their fascist movement," he said.
Another pro-diversity speaker told the crowd UPF's bigoted views should be challenged.
"These fascists, (these) racist bigots, will not march uncontested in our streets," he said.
Before Saturday's protest, Premier Daniel Andrews and Prime Minister Malcolm Turnbull warned against attending race rallies.
Many opponents to the mosque were travelling from interstate to stir up hate and bigotry, Mr Andrews said on Friday.
If people are vilifying another group, "how can that possibly be anything other than contrary to our national interest", the prime minister told reporters in Sydney on Friday.
Australia's Race Discrimination Commissioner Tim Soutphommasane has expressed his disappointment at the rise in anti-Muslim sentiment.
"It's sad that so many police have had to be out in Parramatta and #Bendigo because of fringe groups driven by hatred and division," he tweeted on Saturday.
Federal member for Bendigo Lisa Chesters said the protesters are not representative of the views of Bendigo locals.
"I ask people not to judge Bendigo because the people who are running these rallies are not from our town," she told the ABC. "This is a problem across our country. It is the radical of the left and the radical of the right using Bendigo as the backdrop for the latest stage for their battles."
A bridge now divides the anti-Muslim protestors and the anti-racists @SBSNews pic.twitter.com/ecZu8EWA2Z — Sarah Abo (@_SarahAbo) October 10, 2015
