Ecuador midfielder Caicedo has been attracting interest from a host of the Premier League's top clubs, but is an important part of Potter's side and the club have no interest in letting him go.
Potter, whose side bid to extend their unbeaten start to the season at Fulham on Tuesday night, said: "Well, it doesn't surprise me that people are looking at him because he is playing at a fantastic level.
"As soon as you do research on him you realise he is a top kid, great person, young, playing in the Premier League, has all the attributes to play at the very, very highest level."
Caicedo's value has soared since joining Brighton for £4.5million from Ecuadorian side Independiente del Valle in February 2021.
He made eight Premier League appearances last season after helping Ecuador reach the World Cup finals in Qatar and has played every minute in the top flight this time round as the Seagulls fly high in fourth place.
Potter added: "No surprise, but we know the position he is in and we know the position the club are in, which is why we don't want to lose him.
"We don't think we will, but in football you never know. That is how it is, but we are quite calm and confident."
Potter will hand a late fitness test to Joel Veltman before the clash at Craven Cottage. The Dutch defender was forced off against Leeds after hurting his neck.
Adam Lallana will be sidelined until October due to a calf injury, while Jakub Moder and Jeremy Sarmiento (both knee) are still out.