His bold predictions that he could turn almost anyone into a professional footballer and that graduates from his soccer schools would populate the England team in years to come, were always likely to make waves.
Furthermore, it was always going to be difficult to accommodate so many strong characters at the club with the likes of Redknapp, Lowe, Woodward, Bassett and Wise on the books.
He was appointed on the recommendation of Clive Woodward, who is a big believer in assembling a team of elite coaches to control every aspect of player preparation and he held Clifford in high regard.
This is certainly a setback for Woodward who only a couple of weeks ago in the match programme, was praising the impact his voluntary sessions were having on the youngsters and those on the fringes of the first team, citing the progress of Theo Walcott and the resurgent Neil McCann as among the beneficiaries.
Given that Clifford gave a controversial interview to the local press in the last week claiming among other things that footballers do not train enough, there were always going to be rumours of unrest. There was talk that his amateur background and methods were rubbing up other members of the coaching staff the wrong way. Certainly the timing is suspicious.
Understandably, Saints were showing a united front, with Chairman Rupert Lowe saying that "Simon is a great coach but we needed him here full-time and that was proving difficult. We are just as disappointed as Simon that it has not worked out".
The man that this is likely to have affected most, Clive Woodward, sought to quell the rumours: "I have read some speculation that I am supposed to have sacked Simon; but nothing could be further from the truth. I enjoyed working with him and I still believe he has a great deal to offer the game. It was his decision and I am very sorry to see him go."
A final consideration is for the future of Jason St Juste. A graduate of Simon Clifford's soccer schools the former Darlington player was brought in on a short-term contract. Perhaps his saving grace will be the lack of back-up for the Danny Higginbotham at left-back.
Despite being originally a left-winger, he has put in some encouraging displays at left-back in the reserves and an injury to the ever present Danny Higginbotham may present him with his chance in the first team. The question is whether he will be around long enough to take it with his former mentor out of the club.