Having seen Sergio Aguero put City ahead from the penalty spot after Mehdi Benatia had been sent off at the Etihad Stadium on Tuesday, Bayern hit back through Xabi Alonso's low free-kick and Robert Lewandowski's header.
But Aguero punished a mistake by Alonso to equalise in the 85th minute and then exploited a Jerome Boateng error in injury time to complete a memorable hat-trick, meaning City will qualify if they win at Roma on December 10 and CSKA Moscow fail to win away to Bayern.
"We have to do our duty
It is important," said Pellegrini, whose team are seeking to avoid a third group-stage exit in four seasons.
"We had to play two games -- we won the first one, and now we try to beat Roma away
It also depends what happens with CSKA in Bayern Munich
If they win, they qualify for the next round.
"We will see what happens in the last game
I said last week that if we have a mathematical chance to qualify, we will fight to the end
We will see what happens against Roma."
Aguero has now scored 17 goals for City in all competitions this season and Pellegrini felt that his latest display proved that the 26-year-old deserves to be considered "one of the best players in the world".
Asked to divulge the content of his half-time team-talk, after Lewandowski had put the visitors ahead in the 45th minute, Pellegrini said it had simply been a case of reminding his players what was at stake.
"During half-time, I said to the players that we have 45 minutes more to score two goals and they must fight to the end, because it was our last chance to continue in the Champions League," said the Chilean.
"That is why it has double merit, to beat Bayern Munich, who played very well, but also not to fall down on the spirit.
"Maybe another team would put their hands down, but I don't think this team ever do it
We fight to the end."
- Guardiola 'very proud' -
Bayern were already assured of qualifying as Group E winners, having won their first four games, but beaten coach Pep Guardiola said he was "sad" that his players had not been rewarded for their efforts.
Despite playing with 10 men for over 70 minutes following Benatia's dismissal for a last-man foul on Aguero, the German champions enjoyed 56 percent of possession and completed 563 passes to City's 403.
"We have lost, but I am very, very, very proud of my team," Guardiola told his post-match press conference.
"Playing against the reigning Premier League champions, that is a challenge and we played with 10 men for much of the game.
"Despite that, we dominated the ball and played an aggressive game, but that can happen in the Champions League
It is a group game but if it happens in the last 16 onwards, you are out of the competition."
Guardiola also vowed that his side would not ease up in their final match against CSKA, who need to win at the Allianz Arena to guarantee a place in the next round.
"I think Manchester City and Roma know we are going to try and win the game," said the former Barcelona coach, whose side had previously gone 18 games without defeat.
"We lost, but people saw we didn't come here for holidays or to have fun
We came to win the game and we were so close to doing it."
Source: AFP