Stokes - who will lead England out in two weeks in his first match as Test captain against New Zealand - was looking to continue his fine form against Worcestershire a fortnight ago, where he hit 17 sixes.
However, leg-spinner Hollman took his scalp for just 15 before finishing the day with figures of two for 49, with Toby Roland-Jones claiming two for 40.
Ned Eckersley made an unbeaten 57 for Durham to guide the visitors to 256 for five and leave the game delicately poised.
For much of the morning, Middlesex's decision to bowl first looked a strange one, with the ball barely deviating off straight allowing Alex Lees and fellow opener Michael Jones to make a quick start.
Save for one lbw shout and a loose drive that flew wide of second slip, Lees looked in good touch, hitting four boundaries on his way to 44 before Roland-Jones trapped him lbw.
Jones, who had begun fluently by straight-driving Roland-Jones for four and twice sending Ethan Bamber deliveries to the fence, was increasingly becalmed as the home bowlers dried up the runs.
Reward came soon after lunch when Jones pushed tentatively at Tim Murtagh and inside edged on to his stumps.
Holman then took centre stage, enticing Durham skipper Scott Borthwick to cut one too close to him and lose his stumps.
That brought Stokes to the crease and he signalled his intent to dominate immediately by reverse sweeping the leg-spinner to the third man fence. He swiftly muscled his way to 15, but Middlesex skipper Peter Handscomb stuck by his bowler and was rewarded for his bravery.
Stokes gave Hollman the charge but the all-rounder tossed one a little wider, with the result the lofted drive got more height than distance and Roland-Jones gobbled up the catch at deep mid-off.
At the other end, South Africa international Keegan Petersen nudged and deflected his way into the 40s almost unnoticed, but two short of 50 his first loose shot saw him slash Roland-Jones into the hands of Josh De Caires at backward point.
His departure left Eckersley as the senior batter and the wicketkeeper set about frustrating the hosts. His cover-drive off Roland-Jones just after the second new ball was taken was a candidate for shot of the day and he reached a deserved 50 shortly before stumps.
His partner Liam Trevaskis was granted a life on 21 when Sam Robson grassed a difficult chance low to right at second slip. But, scare survived, he and Eckersley's sixth-wicket stand had realised 73 by the close.