Fury has announced his retirement from boxing on multiple occasions since defeating Dillian Whyte in April and claimed he would only perform a U-turn if he was offered £500million to fight Oleksandr Usyk.
However, the WBC heavyweight champion then indicated he would fight on and sought to arrange a much-anticipated domestic showdown against Anthony Joshua, who has lost three of his last five contests.
While main terms including a 60-40 purse split in his favour were agreed, Fury felt negotiations had to be concluded by Monday at 5pm, but his deadline expired without a deal being done.
He stated that Joshua was now "out in the cold" for a December 3 bout, but Nelson, an ex-boxer and now a high-profile pundit with Sky Sports, believes Fury cannot simply draw a line under the matter.
"It's got to the stage now where both parties have said 'I want this fight' and if this fight doesn't happen, the buck stops with the fighters," said Nelson.
"Will the fight happen? Tyson Fury is slowly losing verbal credibility, because of the things he's saying and not saying.
"You could laugh and joke at it, at first, and say, 'You know Tyson, (it is) good (that) he's not retired', but now the things he's saying are riling the hopes of a nation.
"To say, 'This big fight is happening' and then to say, 'I have changed my mind now, it's not happening because they didn't do it on the day I wanted to do it on'…
"Now there's a backlash coming against him because people are starting to see he's like the boy that cried wolf."
Nelson clarified that he does not believe Fury calling off the fight was in any way out of fear, saying: "I'm not saying Tyson will probably lose to Joshua, or vice-versa. I'm not saying one beats the other."
The pair were close to finalising a fight in 2020 that would have seen all four major world titles on the line, but Fury was ordered to take on Deontay Wilder for a third time by a United States judge.
Joshua lost his WBA, IBF and WBO titles to Usyk last year and was recently outpointed again by the Ukrainian, but the Briton was offered a route back into the world title picture by Fury.
Television executives from BT Sport, with whom Fury works, and streaming service DAZN, which Joshua is tied to, met on Monday to come to an agreement about the broadcast rights for a fight.
The last update from either camp was that discussions were still taking place.