Sergio Aguero scored a 90th-minute penalty winner as City came from behind to beat the Germans 2-1 at a raucous Borussia Park and secure their first points in Champions League Group D.
City, having lost three of their previous four games, had looked set for another night of frustration after falling behind to a Lars Strindl strike early in the second half.
But City rallied, levelling through Nicolas Otamendi and completing the comeback as Aguero, following a night of personal frustration in front of goal, got up after being fouled and struck from the spot.
It was a hard-fought win and City were still some way short of the form that saw them cruise through their opening five games of the season without conceding.
Yet in terms of character, it was encouraging and Hart hopes performances can now improve.
Hart, who himself was outstanding in Germany, said: "We did it the hard way, as our fans know that's how we're doing things at the moment
It wasn't really good enough
We gave far too many chances to them.
"The win is important and we really need to improve, but as badly as we played I think we still deserved to win.
"We need to get back to the first five games of the Premier League, where we were more of a balanced team
We were a bit imbalanced.
"But the season is a long season, so you have got different stages
Hopefully that will put us in a good place for the Newcastle game."
Hart was brilliant throughout as Monchengladbach, rejuvenated under interim coach Andre Schubert after a dismal start to the season, showed as much attacking intent as City.
Hart saved a penalty from Raffael and denied the dangerous Brazilian forward on a number of other occasions during open play.
City also had opportunities but failed to take advantage, with Aguero particularly wasteful as he struggles for form following his recent knee injury.
Strindl's 54th-minute opener came during a period of dominance for the hosts, who were buoyant on their first home appearance in the competition for 37 years.
City responded and levelled through Otamendi after a Martin Demichelis shot was cleared, although replays suggested the initial effort had crossed the line
City then created the better chances in the closing stages and were rewarded as time ran out.
Hart was pleased with his own performance and felt conceding proved the turning point.
He said: "I am enjoying playing
I was gutted to concede the goal but they stopped attacking after that
I didn't touch the ball after the goal and luckily we got the two goals."
Source : PA
Source: PA