Whyte and Povetkin have been put up at the same digs in Brentwood, but Barker says the layout of Matchroom HQ will make a meeting between the two nearly impossible.
The heavyweights both tip the scales at over 200 pounds and sport close to two-metre reach with the Body Snatcher a full two inches taller than the Russian Vityaz.
"It is so small, you walk down the corridors and you are bumping shoulders so I can't imagine those two lumps crossing paths," Barker told Sky Sports.
Promoter Eddie Hearn has staged a series of fights at Matchroom HQ, his former home, and has now added additional measures to keep the heavyweights apart.
"They have actually put a fence in the garden to seperate it so it's going to be interesting," Barker added.
Hearn said that he was considering moving Whyte to another floor because he fears that things could get heated if the fighters bump into each other. It is a matter of conjecture whether Hearn actually believes this or if he is simply hyping the fight.
"Look, we've had fighters in the last three weeks that haven't been the superstar names and they've been a delight to deal with. Next week, the egos come to town," Hearn told Sky Sports."No criticism of the fighters, but we're talking about undisputed champions. We're talking about number one ranked heavyweights. We're talking about Olympic gold medallists.
"And all of sudden the setup changes completely and having spoken to Dillian Whyte I have to raise concerns to the British Boxing Board of Control about people like – and this is my main concern – Dillian Whyte and Alexander Povetkin being in the same hotel.
"Because the issue is not just they're in the same hotel on different floors, they're on the same floor. They're on the same corridor. If Dillian Whyte bumps into Alexander Povetkin in the corridor, or out here on the grass, we can't be responsible for the consequences."
Barker insists that the two heavyweight titans will be on their best behaviour knowing how much scrutiny they will be under.
"I wouldn't advise Whyte to trip Povetkin up or anything like that, I would keep my distance.
"Look, I think they are both experienced fighters out there and I would expect them to behave themselves but anything could happen in there.
"You cannot escape anybody it is that small, so it is going to be interesting."