With the tie heading for extra-time. Payet was brought down on the edge of the box five minutes into stoppage time.
The French superstar picked himself up and curled a superb free-kick into the corner of the net to send the Hammers through into the last 16.
It was Payet's first goal of the season but it came at the perfect time as the victory ended a run of four consecutive defeats.
This has been some summer for Payet following his exploits at the Euro Finals for France and he carried on his good form with another impressive display.
The West Ham fans turned out in their numbers once again to witness another piece of magic from Payet and the 39,877 crowd was the highest League Cup attendance since a fourth round replay against Liverpool back in October 1971.
Slaven Bilic will now be hoping the victory will set his team up nicely for another home game against Southampton in the Premier League on Sunday.
Following the recent defeats against Watford and West Brom you could sense the nerves around London Stadium in the early stages and it was Accrington Stanley who dominated the opening exchanges.
The League Two side could have opened the scoring after just three minutes when Accrington skipper Seamus Conneely fired a low shot which produced a comfortable save from Darren Randolph diving low to his right hand side.
Despite their first ever appearance in the third round of the League Cup, the away side showed they were not just coming to make up the numbers and came close again when Sean McConville raced away down the left hand side and crossed in for Jordan Clark who slid the ball wide of the post.
Accrington continued to create the best openings and had another good chance on 23 minutes when Billy Kee laid the back for John O'Sullivan who fired a shot from the edge of the box just wide of the goal.
The Hammers finally created their first chance on 37 minutes when new signing Edimilson Fernandes, who was making his debut, played a high ball over the top of the Accrington defence. Jonathan Calleri reacted the quickest and lobbed the ball over the bar.
Bilic made a double change at the break in an attempt to turn things around bringing on Manuel Lanzini and Dimitri Payet.
The move almost paid off straight away as Lanzini weaved his magic inside the box and cut the ball back for Simone Zaza who directed a shot straight at Accrington Stanley keeper Elliot Parish.
The danger signs were still there for the Hammers as Accrington came the closest to breaking the deadlock on 55 minutes.
O'Sullivan picked out Sean McConville who was unmarked inside the area and he saw his curling shot produce a superb save by Randolph diving away to his left hand side.
The Hammers raced down the other end as Payet broke free of the Accrington defence and did his best to pick out Zaza and Calleri.
The loose ball fell to Lanzini on the edge of the box and he fired a powerful shot inches over the bar.
The home side were keen to finish the tie inside 90 minutes and had another great chance on 73 minutes when Payet picked out Zaza who saw his close-range shot kept out by the legs of Parish.
The Hammers were reduced to ten men after defender Arthur Masuaku was carried down the tunnel on a stretcher following a challenge late in the game and Bilic had already made his three substitutions.
Pedro Obiang could have won the game in stoppage time at the end of 90 minutes after his dipping shot produced a superb save from Parish.
But no guessing who stole the show with the late winner.
Following his superb set-pieces last season, it was that man Payet who once again became the matchwinner after he was brought down on the edge of the box.
The French star picked himself up and curled an unstoppable shot past Parish into the back of the net to send the Hammers into the fourth round of the competition.
West Ham: Randolph, Arbeloa, Nordtveit, Ogbonna, Masuaku, Feghouli (Payet 46), Obiang, Fernandes, Tore (Lanzini 46), Calleri (Antonio 69), Zaza
Subs not used: Adrian, Byram, Oxford, Fletcher
Bookings:
Accrington Stanley: Parish, Vyner, Beckles, Pearson, Donacien, McConville, Brown (Hughes 76), Conneely (Boco 84), O'Sullivan (Eagles 65), Clark, Kee
Subs not used: Chapman, Gornell, Jones, Lacey
Bookings: Boco
Referee: Stephen Martin
Attendance: 39,877