The Magpie, whose stock has been rising over the last few years, reached his second career European Tour final in Budapest but couldn't quite seal his first continental title.
Joe Cullen put on a dominant display in the final as he claimed the opening five legs without reply.
O'Connor, who missed key doubles early in the match, hit back with double ten and then produced a 124 finish to break with a sparkling 12-darter.
However, five further missed doubles in the next leg allowed Cullen to race to the title.
Earlier in the night, O'Connor came from 5-3 down to edge out Michael Smith in Sunday afternoon's third round, before defeating Jose de Sousa and Nathan Aspinall to reach his first European Tour final since 2018.
"The fantastic crowd in Budapest is one of the best crowds I've ever seen," said O'Connor. "If you told me before the weekend that I'd get to a final, I'd be absolutely ecstatic.
"Although it's a disaster to lose in any final - it's not nice - I don't mind losing to Joe. He's one of the best guys in the game and I wish him the very best in the future."
O'Connor's 2018 European Tour final defeat came against Michael van Gerwen at the European Darts Matchplay.
The Irishman also reached the 2019 World Cup of Darts final, though he once again finished runner-up as Peter Wright and Gary Anderson powered to victory for Scotland.