City, held to a draw by Everton on New Year's Eve, were facing more frustration against an injury-hit yet enterprising Chelsea side at Stamford Bridge.
But less than three minutes after coming on as substitutes with an hour gone, Jack Grealish teed up Mahrez for the winner which cuts the deficit to Premier League leaders Arsenal back to five points.
It also inflicted another defeat on Chelsea, still looking unconvincing under Graham Potter, but who gave City plenty of problems, particularly in the first half despite a double injury blow.
Already without six players, including Reece James and Mason Mount, their evening got off to a bad start when Raheem Sterling was caught by his old City team-mate John Stones in the opening moments and limped off.
Pierre-Emerick Aubameyang was sent on in his place and initially the reshuffle did not knock Chelsea out of their stride, and when Kai Havertz played in Christian Pulisic only a last-ditch tackle from Stones prevented the opening goal.
Potter had said "it never rains but it pours" when discussing his injury list before kick-off but when Pulisic, still feeling that Stones challenge, limped off with only 20 minutes on the clock it became monsoon season for the Blues.
Still they attacked, though, with teenage substitute Carney Chukwuemeka having a shot blocked and Havertz forcing a save from Ederson with the rebound.
City threatened in the latter stages of the first half with Ilkay Gundogan curling a shot wide before he slotted the ball through to Erling Haaland.
The Norwegian looked certain to score when he took a touch with his right foot and prodded goalwards with his left, but the ball flew wide.
Yet Chelsea came agonisingly close to scoring when Chukwuemeka cut inside Kyle Walker and struck a low shot which clanged against the inside of the near post.
Pep Guardiola made a double change at the break, replacing the pair of right-backs he started with, Walker and Joao Cancelo, with Rico Lewis and Manuel Akanji.
It may not have been an attacking change but the difference in intensity from the visitors was palpable.
Haaland dragged another shot wide before Nathan Ake crashed a header against the angle of post and crossbar from Kevin De Bruyne's cross.
De Bruyne then forced the hitherto underemployed Chelsea keeper Kepa Arrizabalaga into a fine save with a low drive from the edge of the area.
Guardiola sent on Grealish and Mahrez on the hour, and the double substitution paid instant dividends.
Haaland fed Grealish down the left and Kepa bafflingly allowed the England winger's low cross to run straight to the far post, where Mahrez had evaded Marc Cucurella to give himself a simple finish.
By contrast to City's, the Chelsea bench looked distinctly callow but Potter still replaced ineffective first-half substitute Aubameyang with teenager Omari Hutchinson.
Their best chance of an equaliser fell to another young substitute, Lewis Hall, who fired over the crossbar in stoppage time.
So City kept the pressure on Arsenal in a title race which will now be on the backburner for FA Cup weekend, upon which these two sides meet again at the Etihad on Sunday.