With the scores level at 1-1 and a replay at Fratton Park looking likely, striker Peter Crouch's injury time goal from the penalty spot booked Saints place in the hat for the fifth round draw.
"I have to hand it to Crouch for volunteering to take the penalty because we had not even decided who should take it," Redknapp said after the 2-1 win.
"I have never seen him as a penalty taker, but he put it away with great authority.
"I thought he was going to miss it and I almost did not want to look but it all came right in the end. This result is sure to give us a lift."
Inevitably the penalty was the major talking point and referee Bennett issued a statement. In it he said: "I was unsighted. I did not see it, but my assistant flagged and when I spoke to him he said it was a penalty and I supported him."
Redknapp admitted: "There were two harsh penalties out there. They were both touch and go. I've seen them given in the past and they could have gone either way.
"Taylor moved his arm to the ball which led to the decision.
"It's been a strange old day. In many ways I have been more excited about a game and I'm glad it's over.
"I'm sorry for Portsmouth. I have no bad feelings about the club or the players. Every one of them came into my office before the game to shake hands and they are a great bunch.
"I brought nearly all of them into Portsmouth at some stage or another and together we had two and a half fantastic years."