Blues boss Potter, who was only appointed in September, is under increasing pressure after his side's poor run continued with defeat at Fulham in the Premier League on Thursday.
New signing Joao Felix was sent off on his debut as Potter's side went down 2-1 at Craven Cottage, their third successive loss and seventh in 10 games in all competitions.
The result also left them 10th in the table, 19 points behind leaders Arsenal having played a game more.
Working as a pundit on BT Sport, Cole said: "He needs time. The fans - we saw a little bit of frustration creeping in here but they've just got to understand there's been so much change.
"The club has been so stable in its own unique way over the last 20-odd years and been successful, but (the new owners) have come in and are changing things behind the scenes, recruitment and have got new ideas about everything at the club.
"I think Graham Potter's acquitted himself very well. That list of injured players - he's got a whole team that would finish comfortably in the top five or six out injured.
"You have to keep going and the fans should stick with him and the players. I believe we've seen enough little sparks of Chelsea to know the results will turn."
Chelsea frequently changed their manager under the club's previous owner Roman Abramovich but Cole would like to see Potter given time to embed a structure.
Cole, a three-time Premier League winner with Chelsea, said: "There's a whole generation of fans that don't usually get this long before there is a change but this is not the way forward.
"This is not what Chelsea have to do. To go and catch Manchester City there needs to be structure, there needs to be foundations and Graham Potter's job is to put all the foundations in. He is the man to steer it in my opinion.
"It's a really tough time but there is no need to panic."