Scott Morrison will meet with Muslim leaders in an effort to tackle extremism, after a deadly terror attack in central Melbourne.
But some leaders who were invited to the private meeting on Thursday have opted to boycott it, after the prime minister told them they should be doing more on the issue.
A group of nine leaders, including Grand Mufti Ibrahim Abu Mohamed, signed a letter on Tuesday night telling Mr Morrison they won't be attending.
"Many in the Muslim community... are deeply concerned and disappointed with statements made by senior government ministers and the prime minister in the recent past, which infer that the community is collectively culpable for the criminal actions of individuals and should be doing more to prevent such acts of violence," the letter said.
"These statements have achieved nothing to address underlying issues,but rather, have alienated large segments of the Muslim community."
Mr Morrison posted a message on social media saying that the meeting would be going ahead, with those who want to seriously deal with the issue "rather than look the other way".
"Some have chosen to publicly boycott this meeting. Continuing down a path of denial only lets their communities down. It makes their communities less safe and more vulnerable," he wrote.
Source: Facebook
Mr Morrison told Islamic leaders they need to do more to combat extremism after a man stabbed another man to death in Melbourne's Bourke Street on November 11.
Labor MP Jason Clare, whose seat of Blaxland has a significant Muslim population, said the leaders should talk to Mr Morrison.
"If people have decided to boycott it I think that is a mistake. Take the opportunity and tell the prime minister what you think," Mr Clare told Sky News.
"Nothing beats dialogue. If you have got a difference of opinion then tell the person face to face, but never walk away from an opportunity to talk to the prime minister of Australia about the issues that you have got."