Luis Suarez scored twice on his return to England as Barca outplayed City during a mesmeric first-half performance at the Etihad Stadium.
Neymar missed a good chance to add a third in the first leg of the last-16 tie and Dani Alves also rattled the bar before the break as the Spanish giants completely overrran the English champions.
But luckily for Pellegrini, his team improved after the break and Sergio Aguero swept home in the 69th minute to give City a goal to take to Barcelona for next month's second leg.
And City were given a further boost when Joe Hart saved a Lionel Messi penalty in injury time, shortly after Gael Clichy was given a second yellow card for a foul on Alves.
"With the way we play, I think we can do it," the City boss said.
"This tie isn't over until the game ends in Barcelona.
"In the second half we demonstrated we are not a lesser team than Barcelona
"It is not the best result here at home
But we are going to play in Barcelona and try and do it the way we normally play.
"It is important not to make the same mistakes
We cannot have a player sent off every game against Barcelona.
"But we have a chance
We must try to do it there.
"It is important for this team to continue in the Champions League as far as possible."
Pellegrini admitted his team were way below par in the first half.
And the Chilean also conceded he owes a debt of gratitude to Hart for saving Messi's penalty, which was awarded after the Barca ace was fouled by Pablo Zabaleta.
"It was a very important save," Pellegrini added.
"It gives us more chance to try and win in Barcelona..
3-1 is a very difficult score to try and erase.
"It was a very unnecessary penalty
I think that we played two different games tonight
"We played very badly until the first half finished
We couldn't put three passes together.
"In the second half I think we played well, we scored one goal and had two or three more chances to draw."
Messi looked extremely annoyed to have missed the penalty that would have made City huge underdogs to qualify.
The Argentinian has been prolific for Barca this year, but he has a poor record from the penalty spot of late.
Messi has missed five of the last 10 penalties he has taken for club and country and has also failed to convert three spot-kicks in the Champions League.
When asked whether he would hand penalty-taking duties to another player in future, Barca boss Luis Enrique said: "No
We know penalties can be missed in this game of ours and he will take the next one
"Tonight is a night to celebrate
All the first half was fantastic from every player, every person on the pitch
"I can't complain about the attitude of the players."
Luckily for Barcelona, Suarez proved far more lethal inside the box than Messi.
Suarez took advantage of some good luck in the 16th minute, putting Barca ahead when the ball kindly fell at his feet after bouncing off Vincent Kompany.
And on the half-hour he doubled the visitors' lead by tapping in Jordi Alba's cross following some masterful trickery by Messi in the build-up.
Suarez was delighted to bag a brace on his return to a country where he divided opinion because of his controversial antics on the pitch.
''I try every game for the team and I think today with (scoring) both goals I am so happy,'' the former Liverpool forward told ITV.
''If they lose the game 3-1 it's really difficult for them in the other game, but I think at 2-1 it's a good result.''
Source : PA
Source: PA