The 2004 Olympic champion - who called it quits after repeating victory over arch-rival Sergey Kovalev at 175-pounds in 2017 - has revealed Anthony Joshua and Tony Bellew were briefly on his hitlist.
Joshua's stock had risen significantly that year after a titanic victory over heavyweight legend Wladimir Klitschko while the Bomber - who starred in the seventh instalment of the Rocky franchise alongside Ward and Sylvestor Stallone - beat David Haye in the first of two bouts against the Hayemaker that year.
The former two-division champion told Joe Rogan in 2019: "I didn't know for sure (if I'd retire).
"But I kind of feeling like, 'Okay, I felt like that after the first one (Kovalev fight), this may be it after the second one.'
"But then the opportunities started coming. HBO had a three-fight deal on the table. And basically, it was going to be a gimme fight in Oakland, at home, kind of a celebration.
"A move to cruiserweight to fight Tony Bellew, probably in the UK.
"There was nothing left at light-heavyweight. So that was the three-fight exit strategy with HBO.
"A gimme fight at light-heavy, I was probably going to do a catchweight at home, cruiserweight. So that was the three-fight deal. And the third fight was going to be Anthony Joshua.
"Don't know if it had been made, we hadn't even gotten that far. But that was our plan - maybe 200lbs," he said when asked about whether he would bulk up to fight at heavyweight.
"I'm never going to be as big as you guys. So why would I weigh myself down? Very dangerous situation. But that was something we were talking about."
He added: "It's all good, though. I've been fighting big guys my whole life."
While the stacks may have been heavily stacked against Ward, he could've been the first super middleweight champion to win heavyweight gold since his hero Roy Jones Jr in 2003.
The American outpointed John Ruiz to claim the WBA belt and became a four-division champion in the process.