Antrim's Allen came from two frames down to lead 4-2 before Williams hit back to force a decider.
Welshman Williams had his chances in the ninth frame but Allen held his nerve to win 5-4.
He will meet Neil Robertson in the semi-finals after the Australian beat Mark Selby 5-2.
Robertson has now won 12 of his last 13 matches against four-time world champion Selby.
Earlier a break of 130 in the deciding frame gave China's Zhou Yuelong a 5-4 victory over David Gilbert.
Zhou will face Anthony McGill, who beat Lyu Haotian 5-3, in the last four.