Parry's two stroke overnight lead went instantly. Kaymer had birdied the first and when he bogeyed it they were locked together at the 12 under par.
They both sank putts of around 15 feet on the long fifth and then Kaymer edged in front again by hitting his approach to within 18 inches of the flag on the next, but at the seventh the US PGA Championship winner made a mess of his second shot and bogeyed.
Parry seized the chance to go ahead again by pitching to 12 feet at the ninth to complete an outward 35.
Lee Westwood, needing to finish first or joint second to replace Tiger Woods as World Number One, was in a tie for seventh on ten under with eight to play.
Westwood parred the first seven before hitting his tee shot to three feet on the short eighth and making a 20 footer two holes later.
Parry three-putted the short 11th - as Westwood did - and Kaymer went back in front with a birdie at the next.
But the 23 year old Challenge Tour graduate matched that by pitching to four feet and they were together again at 14 under, two in front of English trio Gary Boyd, Danny Willett and Simon Dyson - the defending champion - and also Scot Martin Laird.
Parry three-putted again on the 13th, but Kaymer suddenly had different company at the top when Danny Willett, another member of the 2007 Walker Cup side, holed a 50 footer for eagle from off the green at the long 14th.
The German's response was instant, though. His 25 foot birdie putt took him one ahead again on 15 under.