The Citizens came under pressure early on and eight minutes in Jermaine Jones made a fine break down the right, before his cross-shot was superbly touched away for a corner by Joe Hart.
Schalke continued to make most of the running, with City clearly missing the influence of Brazilian ace Robinho, who failed to shake off an ankle knock to feature.
Twenty minutes in Jones again threatened, firing in an excellent 25-yard right-foot volley which was well gathered by the alert Hart.
However, just past the half-hour mark the visitors hit the front against the run of play.
Daniel Sturridge did superbly down the left flank and after skipping away from a couple of markers, his cross somehow found its way past several defenders for Benjani Mwaruwari to fire home at the far post.
City almost made it 2-0 seven minutes before the interval, but after seeing his initial shot blocked, Stephen Ireland was flagged offside as he slotted the rebound home from Sturridge's effort.
Ten minutes into the second half Shaun Wright-Phillips fired in an excellent effort from distance, which forced home keeper Manuel Neuer into an acrobatic tip-over.
The home side for their part continued to try and find a way back into the game, but were frustrated by a well-drilled Citizens backline, being marshalled by skipper Richard Dunne.
With 66 minutes on the clock City made it 2-0 as Ireland exchanged passes with Benjani before beating Neuer to the ball to sidefoot home from 12 yards.
Hart maintained City's two-goal cushion 15 minutes from time as he made an outstanding close-range block from substitute Gerald Asamoah's header, following a free-kick from the right.
That proved to be the home side's last clear chance, as City held out to record a second win in two group games.