The Blues have been linked with several players in the January transfer window as they try and get their Premier League top-four hopes back on track after a poor first half of the season.
They are reported to be in talks with Benfica over a move for World Cup winner Enzo Fernandez, while Andrey Santos, Benoit Badiashile, Joao Felix and Mykhailo Mudryk have all been linked with a move to Stamford Bridge.
Ivory Coast international David Datro Fofana is already through the door following his move from Molde.
Potter would not be drawn on any of that speculation ahead of Thursday's Premier League clash with Manchester City.
"It's a good question but the problem is if I answer it and I'm specific in that answer, every single player in that area will be linked with Chelsea," he said.
"So I have to dodge it - and I hope you can understand why I'm dodging it. I always answer that part of the question by saying that I'm the head coach and my responsibility is to the players that are here.
"We will try in every window to improve, but January is a complicated one because it's the middle of the season. And in the meantime, we have to deal with all the noise of Chelsea getting linked with everybody and I don't comment on any player that isn't a Chelsea player.
"As always, and it's a boring answer, but my job is to help the players that are here and support the club in terms of decisions they make in terms of how to improve."
On Fofana's move, Potter did say: "We still need to wait for the clearances and all those formalities. He's been on a bit of a break so we'll give him some time to train, to assess him, and then work out the best solution in terms of taking the next step in his career."
Potter is wisely concentrating on his current players and how best to improve them after they were fortunate to come away from Nottingham Forest with a point on New Year's Day, a result which left them off the pace in the race for the top four.
Things are not going to get any easier as Pep Guardiola's side visit Stamford Bridge on Thursday, though the defending champions have proved fallible so far this season.
But Potter knows Chelsea will have to be at their best to stop City.
"We have to play well ourselves, that is the key," he added. "Then you have to understand the quality they have and try to stop it. It sounds simple when you condense it down but the reality isn't so straightforward.
"They are a top team but we can cause them a problem, we can be ourselves and try to play well and put them under pressure. We need to use Stamford Bridge and then see what happens.
"They are a top team but we have to think about ourselves, how we can do better, how we can take steps forward in our game. That is the approach we'll have."
Chelsea will still be without right-back Reece James and goalkeeper Eduoard Mendy for the match.
Potter said of the Senegal international: "He's been training a little bit more than he was. He's not fully fit at the moment but hopefully the next week or so he will make progress."