The 24-year-old said he was still in contact with X Factor finalists Saara Aalto and Sam Lavery, but no longer his former flatmate.
Speaking exclusively to the Daily Star Online at the VOXI launch party, Matt said: “I haven’t spoken to Freddy for a while, actually.
“He moved back home. He moved in with me, for a bit. He just found it all a bit difficult and wanted to go back of his family.
“He’s only 18. He’s still a young kid, I guess.”
The ex BFFs were dogged with romance rumours during the X Factor, with fans claiming they saw Matt and Freddy kiss after Terry was crowned winner of the 13th series.
But Matt hit back at the claims they were secretly a couple, with Matt saying the boys were simply enjoying a bromance.
“I don’t think we kissed, but Freddy would have been lucky if we did,” he joked to the Sun. “It is a complete load of rubbish.
“He is my little brother, we hit it off since the start. We are best mates.
“There is no romance. I was just hugging and all of the X Factor contestants were so proud. We are a big family.”
Matt also revealed he still spoke to mentor Nicole Scherzinger “almost every day” and planned to return to the X Factor stage this year.
“I’ll probably hang out with Nicole at the show,” he said. “I’m debuting my new music on the X Factor. It’s coming soon, that’s all I can say about it.”
The singer also teased he may even be a guest at this year’s Judges’ Houses, where Nicole said she’ll strip off “naked and arrive on a jet ski”.
“Wait and see,” Matt said with a wink.
The X Factor returns on ITV on Saturday 2nd and Sunday 3rd September at 8pm on ITV.