The 47-year-old moved to Stamford Bridge from Brighton last month and is so far unbeaten as he prepares to return to the Amex Stadium for the first time on Saturday.
It was during the coronavirus lockdown that Potter's mum died and his dad passed away just six months later, all while the former Birmingham, Stoke and York defender was cutting his teeth as a Premier League boss at Brighton.
Now he has moved on to Chelsea and believes comparisons to his managerial predecessors can be a catalyst for mental ill health.
"Before I went to Brighton, I had no experience as a Premier League manager so I learnt that I could do that," he replied when asked for the biggest lesson he took from his three years at the Amex Stadium.
"Sometimes you have to suffer and you have to experience pain along the way and obviously the higher you are in the Premier League the more noise there is.
"You (the media) want to compare with the previous guy so there is always that going on. Which isn't great for the mental health if you're comparing, I would say generally.
"My mum and dad passed away during that first period of time so then you're going 'OK I've got these feelings here that are powerful and raw' but then you're also trying to compete in the Premier League so then you're trying to manage - 'OK, am I getting angry or disappointed or frustrated or whatever the feelings are because of this or because of this?'
"So that was the first six months of life in the Premier League and I think the quote is 'you're fixing the plane while it's up in the air.' That's a great quote."
Potter became clearly emotional as he spoke about the death of his parents, pausing slightly as he explained how much his father would have loved to see him managing Chelsea in a Champions League game at the San Siro.
"We are part of a sport where we create pressure," he said when asked if the mental health of managers needs to be brought into the open.
"Somebody has to be under pressure, whatever it is and it will be one after the other, after the other, after the other.
"Then one's gone and it is on to the next, it was Steven Gerrard a few weeks ago and then it will be somebody else and then somebody else.
"It is difficult in the world that we are living in to feel sorry for a Premier League manager, get me right, but mental health doesn't really discriminate with your status or how much money you earn either, I would say.
"It is just something to be aware of, it is a challenge and I think we all have to be mindful of that. I think you have to understand you do the job and there are things out of your control that you have to manage and you have to deal with."
An early test of the stresses of management came Potter's way when he decided to uproot his young family to take a managerial post at Swedish lower-league outfit Ostersunds over a decade ago.
He took the club from the fourth tier to the Europa League before heading back to England, admitting it was a tough time in a small Swedish town.
"I suppose when you move to Sweden in the northern part of the country and it's -20 outside in the winter and your wife has left everything that she knew, she is there with an 11-month-old kid, crying, because she misses her family and her job; then you sort of think, I have got to make this work," he said.
"Then by definition you throw yourself into it and maybe it becomes a bit of a habit, I don't know. It's difficult, you have to work hard if you want to achieve anything. You have to work hard."
Spending time with his family is how Potter revealed he escapes the pressure of managing in the Premier League, that and a "brain-dump" watching of a television box-set.
"Vikings: Valhalla I think it was called," he said was his latest binge.
"My Mrs would go mad because it was just blood and guts!"