Nick Watney was three clear with a round to go, but double-bogeyed the opening hole and triple-bogeyed the short seventh after pushing his tee shot into the rocks and from there into Lake Michigan.
With 21 year old Northern Irishman Rory McIlroy managing only a one-over 37 to the turn - it could have been much worse and he had kept himself in it - Kaymer's birdies at the second, fourth and tenth made him the man to catch.
The 25 year old was 12 under par, but McIlroy's matching birdie on the tenth brought him back alongside former winner Steve Elkington, now 47, and big-hitting American left-hander Bubba Watson in second place.
McIlroy, who resumed three behind Watney, scrambled well early on, but then bogeyed the fourth following a bad drive.
He got that shot back on the seventh, but bogeyed the next.
Watney, all the way down to joint 11th, also bogeyed the fourth, eighth and ninth in a nightmare outward 43.
His playing partner Dustin Johnson had every sympathy. He was three in front after 54 holes of the US Open Championship in June and shot 82 to let in Northern Ireland's Graeme McDowell.
Johnson this time retained his hopes of making amends by matching McIlroy's one over effort going out.
Watson's length came in handy on the 569 yard 16th and he used it to great effect, reaching the green in two and two-putting for a birdie that brought him level with Kaymer on 12 under.
They were two ahead of McIlroy, Elkington and Johnson.
Watson went down the cliff edge left of the 17th green and bogeyed, but as that happened Kaymer missed the 15th green, chipped to seven feet and missed.
That brought McIlroy into the joint lead as he had just made a ten footer on the 14th, while Elkington and Johnson were only one back.