Smith carded a flawless final round of 64 - his second such score in the space of three days - to overturn a four-stroke deficit and win by a shot from playing partner Cameron Young.
The 28-year-old's total of 20 under par surpassed the previous best for an Open on the Old Course, set by Tiger Woods in 2000, by one and sealed a third win of the season for the new world number two.
Smith started the year with a record 34-under-par winning total in the Tournament of Champions in Hawaii following a thrilling duel with then world number one Jon Rahm.
Two months later he won the prestigious Players Championship at Sawgrass and Sunday's victory at St Andrews means he joins Nicklaus as the only men to win the Players and Open in the same season.
Nicklaus, whose double was also sealed at St Andrews in 1978, posted on social media: "Wow, what a back 9 of golf and putting at @TheOpen by Cameron Smith! Just fantastic! That was fun to watch.
"My congratulations to the Champion Golfer of the Year, Cam Smith, on winning his first major and at no better place than the Home of Golf, the Old Course at St Andrews!"
Nicklaus, who had been in St Andrews to be named an honorary citizen of the town, also praised the "tremendous performances" of runner-up Cameron Young and third-placed Rory McIlroy, who had shared the 54-hole lead with Viktor Hovland.
"Really, @McIlroyRory played well all day but never one-putted a green, while Cam had 6 on his back 9 alone," Nicklaus added.
"Cam Smith showed what makes the difference in a tournament, especially a major - who makes the putts coming down the stretch. 30 on the back 9! 64 on Sunday! Unbelievable!"
Smith's previous best finishes in majors both came in the Masters as he finished runner-up at Augusta National in 2020 and was third in April.
In 2020, with the tournament held in November due to the Covid-19 pandemic, Smith was one shot behind Dustin Johnson after nine holes of his final round, but could only play the back nine in level par to finish five shots behind the winner.
In April, Smith birdied the first two holes in the final round to cut Scottie Scheffler's lead to a single shot, but bogeyed the third and fourth and saw any hopes extinguished with a triple-bogey on the 12th.
Asked if improving mentally had been crucial to making his major breakthrough, Smith said: "I don't think I've changed a lot mentally.
"I think sometimes you get away from what you're doing, and I think it's just a thing of just getting back to what you know and what you know works.
"I've definitely been on that track a few times in my career. But I think it's just honestly belief. The Players at the start of the year, with the best field in golf, to go away with the win was a really big confidence booster.
"I knew it wasn't going to be too long before I got one of these. I've knocked on the door, I think, maybe one too many times now. So it's nice to get it done.
"All the names on the Claret Jug, every player that's been at the top of their game has won this championship. It's pretty cool to be on there. It really hasn't sunk in yet. I don't think it will for a few weeks. It's just unreal."