The Latics are in the bottom half of League One while City entered the third round tie after drawing 1-1 with champions Arsenal, yet status went out of the window on a day of typical of cup action.
The conditions were poor with a gale force wind hindering both sides while a sell-out crown was in raucous mood from the start.
And it was Oldham fans celebrating a famous win after their side held on to Vernon's early strike for the Latics.
Oldham thought they were ahead after just four minutes but referee Mark Dean spotted a foul on City keeper David James as an inswinging corner from David Eyres went straight past him.
Joey Barton fired wide for the Premiership club, but Oldham scored what proved to be their winner after just 14 minutes. They were awarded a free-kick on the right and Neil Kilkenny's ball to the far post was knocked back across by Eyres.
Vernon got away from the City's defence ten yards out and calmly planted his right-foot shot inside the post.
Vernon tested James with a half-volley before Oldham veteran Eyres headed off the line from Shaun Wright-Phillips on 42 minutes.
Oldham wasted a chance just after half-time as Kevin Betsy fired off target, but it was backs to the wall after that as City piled forward in numbers.
Home goalkeeper Les Pogliacomi was in superb form as he denied Antoine Sibierski and Bradley Wright-Phillips before the other Wright-Philips, Shaun, was kept out by a flying save.
City pressed until the end but the Oldham defence was in defiant mood, with Will Haining and Danny Hall superb at his heart of the backline.
Man of the Match: David Eyres - He may be almost 41 years of age, but the left-sided midfielder made vital contributions at both ends to ensure an FA Cup shock.