Two goals in the final 20 minutes from Ross Hamilton and Robert Sloan gave new manager Allan Maitland a winning start.
Livingston led at half-time thanks to a Paul Dalglish penalty, but Alloa refused to accept defeat as they took the game to their opponents after the break.
Alloa troubled Livingston throughout an untidy match in which they showed they were far from overawed as they booked their fourth round place.
Sloan had a couple of decent efforts on goal in the first half which Roddy McKenzie had to watch closely.
There was an early scare for the lower division side when goalkeeper James Evans allowed a long through ball to sail over his head but the ball eluded Dalglish.
Livingston had to wait until seven minutes from half-time to get the breakthrough goal.
Fraser Forrest made a careless challenge on Dalglish in the penalty area and the Livingston striker dusted himself down to slot home the resultant penalty.
Dalglish had another chance shortly before half-time from a Dave Mackay pass but pulled his shot just wide.
Dave McGeown had a chance to equalise shortly after half-time but could not direct his header on target.
Livingston sought to kill off the game and a tremendous free-kick from Robert Snodgrass was just a yard too high.
But Alloa fought back and Hamilton equalised with 20 minutes left when he glanced a header past McKenzie from a Jamie Stevenson corner.
It gave Alloa fresh heart and nine minutes from time Sloan struck the winner with a fine shot which left McKenzie with no chance.