Erik ten Hag's side were on course to go second in the Premier League table after Bruno Fernandes' first-half opener.
But Olise's late strike means they now head to leaders Arsenal on Sunday eight points adrift and missing key midfielder Casemiro, who will be suspended after picking up a fifth booking of the season late on.
United debutant Wout Weghorst nearly opened his account for his new side, directing a first-half header over the crossbar, while a brilliant stop by David De Gea denied the hosts from taking a first half lead.
Fernandes handed the visitors the lead just before half-time, and United seemed poised to make it 10 wins in a row in all competitions before Olise levelled.
United dominated possession early on but failed to create clear-cut chances, with the in-form Marcus Rashford - who had scored in all seven games since returning from the World Cup prior - being kept quiet by the home defence.
Marc Guehi did well to stop the England forward from converting a dangerous run into an opener and by the 10-minute mark the visitors had managed just a single shot from Fernandes, dangerously spilled by Vicente Guaita before his defenders cleared.
Luke Shaw nearly opened the scoring but directed his powerful strike just wide of Guaita's right post, while United survived a scare when Lisandro Martinez appeared to be caught by an elbow and lay on the pitch while he was attended to by medical staff.
He was deemed fit to continue and returned to action wearing a band over his head.
Weghorst nearly opened his account for United when he rose to meet Luke Shaw's cross from the left, but the Dutch striker nodded just over the crossbar.
United went close again when Rashford directed a 39th-minute free-kick wide.
The Eagles were inches away from taking the lead when Olise found Odsonne Edouard, who directed a strike towards the top left corner but was denied by a brilliant leaping save from De Gea, who got his fingertips on the ball before it touched the crossbar.
And soon after the visitors broke the deadlock, when Christian Eriksen picked out Fernandes, who took a touch before firing past Guaita to finish a well-worked move in the 44th minute.
Patrick Vieira made his first change in the 56th minute and, as Eberechi Eze came on for Jean-Philippe Mateta, a pitch invader took advantage of the pause and somehow made it as far as midfield, where he appeared to take a selfie with Casemiro before belatedly being escorted off.
United wanted a penalty when Chris Richards broke up Scott McTominay's run, but a VAR check left the hosts unscathed.
Palace then went on the attack, with Guehi's low header blocked by De Gea.
The hosts kept pushing for an equaliser as Olise flicked a pass to Nathaniel Clyne, whose cross into the six-yard box evaded all his team-mates.
Casemiro had avoided a booking until he was punished for a challenge on Wilfried Zaha in the 80th minute.
That seemed to be the extent of the hosts' woes until Olise brilliantly curled his free-kick into the top corner.
Both sides had chances for a late winner, with Casemiro almost getting on the end of a corner at one end before Aaron Wan-Bissaka's superb challenge denied Zaha at the other.