Despite being named on the SPOTY six-person shortlist earlier this week, Fury will be spending his time in Florida rather than a Salford studio, and won't be answering any calls.
The world heavyweight champion had previously threatened legal action if he was included, but probably couldn't find a lawyer willing to take on that unusual case.
His 2021 nomination comes on the back of a second crushing defeat of Deontay Wilder, which ended their trilogy of title fights.
Emma Raducanu (1/33), Adam Peaty (100/1), Tom Daley (14/1), Raheem Sterling (125/1) and Dame Sarah Sarah Storey (150/1) are the other athletes in contention for the Award, which will be presented in front of a drastically-reduced audience in Salford on Sunday.
Fury told the Daily Mail: "The BBC would never let me win even if I knocked out King Kong.
"I will be relaxed in the Florida sunshine while the BBC get on with their SPOTY in winter. Even if they did want to give me another Special Award, which they did once, I won't be making another speech for them."
Fury finished fourth in 2015, despite a petition calling for his removal from the shortlist after comments he made linking homosexuality with paedophilia surfaced, and was also shortlisted last year.