Murray was speaking after Great Britain fell to an agonising defeat against the USA in their Davis Cup tie in Glasgow.
Dan Evans' defeat against Tommy Paul and Cameron Norrie's incredible victory over Taylor Fritz had set the stage for a deciding doubles contest, which only started after 10pm at the Emirates Arena.
Captain Leon Smith picked Murray alongside world doubles number one Joe Salisbury for the clash against Jack Sock and Rajeev Ram.
Murray and Salisbury looked to be in control leading by a set and a break but the American duo fought back strongly and eventually clinched a 5-7 6-4 7-5 victory at 12.58am.
A sizeable crowd did stay until the end but a start time of 4pm always seemed risky and, with the last trains long having departed, many fans had to leave.
"It would be better if they were earlier I think for everyone involved. I don't think it's ideal for the fans," said Murray.
"That match that we played there, there's probably half of the people in there at the end of the match that was there at the beginning. It's a bit of a shame because they missed a great match. It's not probably because they didn't want to stay. If you got children here, you can't stay with them. You got to get a bus or train home. Can't do it."
Murray also said the situation was far from ideal for the victorious Americans, who will have very little time to recover before playing Kazakhstan on Thursday.
"They're (Team USA) probably not going to be tucked up in their beds until, I don't know what the time is, probably 3:30, 4:00 in the morning by the time they've done their recovery and managed to sort of calm down after a tie like that," he said.
"It's not just here. We've seen it obviously at the US Open even just last week. It's something that tennis needs to sort of have a bit of a think about. I don't think it looks that professional."
Great Britain will likely need to win their next two ties to secure qualification for the Davis Cup finals in November, something captain Smith admits will be a "tough" task.
"It's still going to be tough because the other two nations are strong. I think a lot of the matches are 50/50, including the ones that are coming up," he said.
"All the guys played really good matches, so it will take a lot of recovery tomorrow (Thursday). It's obviously good that we've got a day's grace between our next match."
Great Britain will play the Netherlands on Friday and then Kazakhstan on Sunday.