Rovers led through an early goal from Jack Vale before efforts from Pablo Fornals and Michail Antonio turned things around in West Ham's favour.
Substitute Ben Brereton Diaz then netted an 88th-minute equaliser to send the third-round tie to a shoot-out and, after 19 successful spot-kicks, Angelo Ogbonna hitting the bar saw the Premier League side knocked out.
Two top-flight outfits were taken to penalties by League One sides but made it through to round four, in defending champions Liverpool and Southampton.
Liverpool need penalties at Anfield
Liverpool's defence of the Carabao Cup began the way it was won with a penalty shoot-out victory in which goalkeeper Caoimhin Kelleher played a pivotal role.
The Republic of Ireland international, playing his first game since scoring the winning spot-kick against Chelsea at Wembley in February, saved three Derby penalties as Jurgen Klopp's untried side scraped past their League One opponents after a goalless draw.
Harvey Elliott, introduced as a 65th-minute substitute along with Roberto Firmino and Darwin Nunez, scored the decisive kick as the disappointing hosts won 3-2 in the shoot-out.
That it had reached that point was perhaps no surprise as Liverpool, who registered 11 changes from Sunday's win over Tottenham, have not won a League Cup tie in normal time at Anfield since a 2-0 victory over Leeds in November 2016.
Full debuts were handed to summer signing Calvin Ramsay (19), 17-year-old Bobby Clark - son of former Newcastle midfielder Lee - 18-year-olds Stefan Bajcetic and Melkamu Frauendorf and Layton Stewart (20).
Southampton get past Sheffield Wednesday
Southampton got past Sheffield Wednesday 6-5 after a 1-1 draw at St Mary's, watched from the stands by prospective new Saints manager Nathan Jones.
The hosts, led by interim boss Ruben Selles following Ralph Hasenhuttl's sacking, fell behind to a Josh Windass goal before James Ward-Prowse equalised with a penalty on the stroke of half-time.
Alex McCarthy then saved Dominic Iorfa's spot-kick in the subsequent shoot-out to put Southampton through.
Arsenal crash out against Brighton
A much-changed Arsenal side crashed out of the Carabao Cup at the hands of Brighton as debutant goalkeeper Karl Hein had a night to forget.
The Gunners sit top of the Premier League and qualified as winners of their Europa League group, but Mikel Arteta made 10 alterations for the tie and Brighton came from behind to win 3-1 at the Emirates Stadium.
Eddie Nketiah finished off a fine move to put the home side ahead before Hein fell over and conceded a penalty all in one motion as former Arsenal man Danny Welbeck levelled from the spot.
Two breakaway goals after the interval, firstly through substitute Kaoru Mitoma and then Tariq Lamptey, saw Brighton advance to round four.
It was a first home defeat in 12 games for Arsenal, the last also coming at the hands of the Seagulls, who have now lost just one of their last five visits.
Manchester City cruise past Chelsea
Riyad Mahrez and Julian Alvarez struck early in the second half as Manchester City began their latest Carabao Cup quest with a 2-0 win over Chelsea.
Mahrez gave City the lead with a fine free-kick after 53 minutes of Wednesday's third-round tie before Alvarez tapped in the second five minutes later at the Etihad Stadium.
Chelsea's impressive youngster Lewis Hall twice brought good saves out of Stefan Ortega but City, bidding to win the competition for a seventh time in 10 seasons, were good value for their victory.
Further good news for the home side - and potentially England - was that midfielder Kalvin Phillips had a 40-minute run-out.
Phillips had been out since undergoing shoulder surgery in September and the game was his last opportunity to prove his fitness before England boss Gareth Southgate names his World Cup squad on Thursday.
Nottingham Forest see off Tottenham
Jesse Lingard finally broke his Nottingham Forest duck as he scored in a 2-0 win that dumped sorry Tottenham out of the Carabao Cup.
Lingard has had an underwhelming start to his Forest career after his surprise free transfer in the summer and had not scored or made an assist in his opening 12 appearances - but he put that right on a memorable night at the City Ground.
The former Manchester United forward headed home from close range in the second half, seven minutes after Renan Lodi curled in to give the Reds a deserved lead on a humiliating night for Spurs.
This competition was a credible chance for Antonio Conte's men to end their trophy drought but this was a dismal display.
Traore hands Wolves win against Leeds
Boubacar Traore's late winner sent Wolves through to the last 16 of the Carabao Cup.
The substitute struck with five minutes left to down much-changed Leeds and seal a 1-0 victory for the hosts.
It settled an uneventful game which looked to be heading to penalties and gave new Wolves boss Julen Lopetegui and his coaches food for thought.
Lopetegui is due at the club's Compton training base on Friday, ahead of watching Saturday's visit of Arsenal, before officially taking charge on Monday.
The former Spain manager's lieutenants were at Molineux having already briefly introduced themselves to the squad and they would have been concerned and encouraged in equal measure early on.
Pope stars for Newcastle
Nick Pope enhanced his World Cup credentials with three penalty shoot-out saves as Newcastle edged past Crystal Palace into the fourth round of the Carabao Cup.
The night before England boss Gareth Southgate named his squad for Qatar, the Magpies keeper denied Luka Milivojevic, Jean-Philippe Mateta and Malcolm Ebiowei from 12 yards to secure a 3-2 shoot-out victory after a drab 0-0 draw on Tyneside.
Thursday sees round three conclude with Manchester United hosting Aston Villa and the draw for the fourth round taking place.