Three quarters of Australians believe the country doesn't need any more people and nearly half support a partial ban on Muslim immigrants.
An Australian Population Research Institute survey of more than 2000 people also found 54 per cent want a reduction in the annual migrant intake.
The organisation's researchers, Katharine Betts and Bob Birrell, say the result shows a disconnect between the political elites' commitment to high immigration policies and the concerns of voters.
In their analysis, they said the results are driven by the impact of population growth on people's quality of life and the rapid change in Australia's ethnic and religious make-up.
"Australian voters' concern about immigration levels and ethnic diversity does not derive from economic adversity," academics Katharine Betts and Bob Birrell wrote in a report based on the survey.
"Rather, it stems from the increasingly obvious impact of population growth on their quality of life and the rapid change in Australia's ethnic and religious make-up."
Related reading
LNP rejects limited Muslim immigration ban
Australia's population increased by 389,000 people to 24.5 million in the year to March, largely due to the arrival of new immigrants.
Most people who migrate to Australia are skilled workers (68 per cent) and about a third make the move to be with family.
But 74 four per cent of those surveyed believe Australia is "already full", with most pointing to roads congestion, hospitals capacity, affordable housing and fewer jobs as evidence.
Mr Birrell told SBS News that "population pressures" significantly contributed to this result.
"For most Australian voters, the problems associated with Australia’s very high population growth, which is higher than other developed countries are now starting to bite," he said.
"We’re seeing that in our survey that most respondents thought that population pressures were adding to difficulties of urban congestion, housing affordability, job competing and the like. It's hardly surprising that 74 per cent of them would respond by saying Australia doesn’t need more people."
Some 54 per cent want Australia to cut its annual immigrant intake of about 190,000 people and 48 per cent backed a partial ban on Muslim immigration.
However, another 27 per cent were undecided about a partial ban, while a quarter opposed it.
The strongest support for the partial ban came from One Nation voters (89 per cent), with more than 50 per cent of Liberal voters agreeing and just over a third of Labor supporters.
"The willingness to take a tough, discriminating stance on Muslim immigration is not limited to a small minority, but extends to almost half of all voters," the report said.
Immigration minister Peter Dutton reacted cautiously to the survey results on Thursday, telling 2GB's Ray Hadley that the government was "always looking at the migration numbers" to get the balance right.
"In the Labor years the number peaked at about 305,900 in one year which was an enormous number, we've got that number down now below 190,000 and as I say, we’re happy to reassess."
He said new migrants were drawn to the big population centres where pressure on housing and infrastructure was most often felt, however, "In some regional towns they’re crying out for people because they can’t get workers in the meatworks or areas of primary production, tourism, restaurants and so on. So we need to get that balance right."
Mr Dutton linked the issue to his efforts to reform citizenship laws, which have been struck down in the Senate.
"We want to make sure that people who want to become Australian citizens … have integrated into Australian society, that are abiding by our laws and ahering to our values."
More than half of those surveyed feared Australia risked losing its culture and identity, with a similar number saying it had changed beyond recognition and sometimes "felt like a foreign country".
Related reading
Immigration Minister stands by Armani refugees comments
Australia's political and economic "elites" had ignored rising concerns about immigration, the report said while noting rising support for anti-immigration parties across Europe.
"Such is the extent of these concerns that they could readily be mobilised in an electoral context by One Nation or any other party with a similar agenda, should such a party be able to mount a national campaign," the report said.
"If this occurs, the Liberal Party is likely to be the main loser."
The survey was largely based on the views of Australian-born respondents who were "much more likely to take a tough line on immigration numbers and ethnic diversity than are overseas-born persons (unless they are UK-born)", the report noted.
The institute commissioned the 10-minute survey from July 31 to August 17 this year, where a random national sample of 2067 voters, drawn from an online panel of 300,000 people, were asked questions about Australia's immigration and population policies.
The survey's results conflict strongly with research by the Scanlon Foundation a year earlier which found just 34 per cent of Australians thought immigration was too high.
The researchers conceded that different methodologies may have led to the significantly different results, but said "more likely... the recent increase in media and public attention to the immigration question has contributed to a hardening of voters' attitudes".
