In her Olympic debut, Brown was aiming for a medal on Wednesday and she had an impressive ride of 31 minutes 22.22 seconds over the 22.1km course.
That gave her the second-fastest time behind van Vleuten's storming 30:13:49, with four other riders starting after Brown.
First Marlen Reusser posted 31:09:96 for the silver and then van Vleuten's compatriot Anna van der Breggen, the reigning world time trial champion, took bronze with 31:15:12. Young Australian Sarah Gigante again showed she is a rising star with her 11th-placed time of 33:01:60.
"I got every little bit out of myself and nothing left to spare, so I'm happy with myself despite being so close to a medal," Brown said.
"No matter the result, I think after I passed the finish line I knew I had left nothing there. I was in the flow, I was in the zone, and that's usually the feeling I want to have," Van Vleuten told reporters.
The Dutch winner has strong Australian connections, having ridden for five years with professional team Mitchelton-Scott before switching this season to Movistar.
The gold medal comes five years after van Vleuten had a horrific crash on the last descent while on the way to winning the Rio Games road race. The Dutch rider suffered three spinal fractures and a severe concussion.
She was back competing only weeks later and was one of the top riders coming into Tokyo's road events, winning the 2017 and '18 world time trial championships and the 2019 world road race title.
There was another Olympic setback on Sunday in the women's road race, when van Vleuten crossed the line in triumph, thinking she had won.
But van Vleuten and several other top riders did not realise that Austrian Anna Kiesenhofer had won the gold medal 1:15 earlier. This time, there was no mistake as van Vleuten scorched around the undulating course.
