The grave was found in Hamam al-Alil area, about 14 kilometres (8 miles) from the southern outskirts of Mosul, the last IS-held Iraqi city.
"Federal police found a mass grave west of Hamam al-Alil in the agricultural college," the police said in a statement.
They released a series of photos showing security personnel and other people in civilian clothes in an open area of disturbed, uneven earth littered with trash.
Some of the images showed indistinct shapes that may be bodies buried in shallow graves among the garbage.
The Joint Operations Command said that "Iraqi forces found... 100 bodies of citizens with their heads cut off" at the college, and that specialised teams would investigate.
Related reading
Syrian militias make progress in IS-Raqqa
It was unclear how the JOC arrived at that figure or determined that they had all been beheaded given that the bodies at the site appear to still be buried.
Iraqi officials have previously estimated the number of victims in mass graves before they have been excavated and counted.
IS overran large areas north and west of Baghdad in 2014, declaring a cross-border "caliphate" that also included territory in Syria.
Related reading
IS hits back in Mosul as Raqqa front opens
Its rule has been marked by repeated atrocities including mass beheadings and other executions that it has documented in photos and videos lauding the violence that its supporters share online.
Iraqi forces have since regained much of the territory that IS seized, and have uncovered a series of mass graves and massacre sites as they have pushed the jihadists back.
Iraqi forces are now fighting to retake Mosul from IS, and Hamam al-Alil was one of the last major obstacles between them and the southern outskirts of the city.
