Things got off to a promising start for the Black Cats as they went in front after just nine minutes.
David Meyler spread the ball out to Steed Malbranque on the left and he cut inside to curl a perfect cross into the area. Striker Kenwyne Jones got up above the City defence to head past Shay Given and into the net.
Five minutes earlier Jones had a chance to open the scoring, but fired straight at Given.
Sunderland were the more productive going forward, with City's moves breaking down at key moments. They did manage to get into the danger area on 14 minutes, but appeals for a penalty were waived away as Micah Richards went down in the area.
Jones was clearly enjoying himself and tried an audacious lob from distance on 20 minutes, which came to nothing.
City's best chance of the opening 45 minutes came just after the half-hour mark, as Gareth Barry curled his shot wide.
Sunderland were forced to ring the changes at the break, with young midfielder Jordan Henderson replacing goalscorer Jones, who appeared to have picked up a knock towards the end of the first half.
Jones was a miss after the break, as City who had been troubled by the big Trinidad & Tobago frontman, started to get on to the front foot.
But the hosts could have increased their lead on 49 minutes. Darren Bent backheeled the ball in to Fraizer Campbell, but his chip was comfortably gathered by Given.
A minute later City were on the attack as Carlos Tevez put in substitute Roque Santa Cruz, but Craig Gordon made a great save.
It was to be the first of many from Gordon, who made three top saves in as many minutes to deny Craig Bellamy twice and Shaun Wright-Phillips.
Gordon set to be City's nemesis as he denied them time and time again, with Bellamy in particular finding the Scot up to everything he threw at him.
But right at the end the visitors found a way through, Gordon saved from Bellamy again, but from the resulting corner the ball dropped to Johnson, who curled a superb left-foot effort into the top corner.
It was ecstasy for the away side and agony for the hosts, but overall Steve Bruce will have been happier with the point earned, while the visitors missed out on a chance to move into the top four.