The Senegal captain will help shore up the Blues defence this season following the departures of Toni Rudiger and Andreas Christensen.
Koulibaly asked former Chelsea legend Terry for his blessing to take the number 26, although he admits it took a while for the England defender to believe who he was.
"I called John and at the beginning he didn't believe it was me, he thought it was a joke. So he called the team manager to ask if it was really me," said Koulibaly.
It will come as welcome news to Chelsea fans that the 31-year-old Koulibaly had looked into the history of the West London club and was keen not to create an unwitting PR disaster by taking a shirt which played an important part of the club's recent success.
"I wanted to know which number was free, I was sent some numbers and I wasn't sent the 26," said Koulibaly.
"I asked for 26, and they said John left it in 2017 and nobody took it since then.
"So I was asking myself, 'is it retired or does nobody want to take it?'.
"So I asked Gianfranco Zola, because I know him very well and have been asking him so many questions.
"I asked him for John's phone number because I wanted to ask him about the jersey.
"I called John and at the beginning he didn't believe it was me, he thought it was a joke. So he called the team manager to ask if it was really me.
"I know that it's a very important number for John, but it's also a very important number for me too, because I took it at Napoli.
"And when he said yes I was very happy, because I know what he did for the club and for the supporters here.
"He give me his answer and I was happy, so I wanted to tell everybody that I ask him before, because I didn't want to disrespect a legend of the club."
Koulibaly, meanwhile, has vowed he will never let anyone stop him representing Senegal.
The Senegal captain called for more respect of African nations after the president of his former club Napoli threatened only to sign African players willing to give up international ambitions.
Aurelio De Laurentiis pledged that Napoli will only now sign African players if they are willing to miss the Africa Cup of Nations, given the tournament clashes with European league competition.
Koulibaly has joined Chelsea in a £34million deal from Napoli, becoming part of new owner Todd Boehly's new Stamford Bridge era.
The 31-year-old insisted he would respect De Laurentiis' pledge on African players, but urged European teams to do likewise to the Africa Cup of Nations.
"Nobody ever told me not to go to AFCON or something like this," said Koulibaly, who captained Senegal to AFCON glory at the 2022 tournament. "Maybe sometimes they try to ask my manager or something like this.
"But me, now as captain of Senegal, when AFCON comes I have to be the first one there, the first one in Senegal.
"Nobody can tell me not to go to my national team. I have a lot of love for my national team, the team that I play for.
"So, if someone told me to do that, it would be the only time that I can fight with somebody.
"The way we feel all the time with my national team, we are all respectful, we wait for players to come from the Premier League, and everybody is happy to be here.
"Everybody thinks the same as me in the national team. If someone told them not to come they would always fight to come.
"This is the most important thing. Everybody has to give us respect."
Napoli chief De Laurentiis this week lamented the time African players spend away from the Italian season competing in tournaments like AFCON.
"We are the idiots who pay salaries only to send them all over the world playing for others," he said.
When asked for his opinion on his former club president's views, Koulibaly said: "It's up to him, if he wanted to say this.
"But for me the most important thing is to respect everybody.
"When I played there I was playing also for Senegal. I won AFCON when I was playing for Senegal for Napoli.
"It's true it's difficult for them, but I think you have to respect the national team also.
"You have to have respect like you have of European national teams.
"As captain of Senegal it's not a good way to speak about African nationality, but I respect what he thinks.
"If he thinks that, it's up to him, but I think not everybody has the same idea as him in the club. The supporters don't think like this.
"It's up to him, it's not an idea of the society or the city, the city is respectful of everything.
"This world, it's what he thinks, not what the team thinks or what the city thinks."