Alloa, after successive draws, showed tiredness after their midweek exertions and finally caved in under tremendous pressure.
Alloa made a flying start when James Stevenson's free-kick from 30 yards bounced awkwardly and goalkeeper Gary O'Connor saw it slip under his body into the net.
O'Connor redeemed himself later with a great save from another Stevenson's piledriver.
In the second half Berwick built up the momentum and James Evans made a fine save from Darren Smith after the winger had beaten three defenders.
In the 73rd minute, with Paul Hampshire on the touchline waiting to replace him, Smith hammered the equaliser after Kevin Gordon's shot hit the post.
As the pressure and excitement mounted, Alloa defended with determination but Rangers kept going and forced a corner on the left.
David Murie took it, Gareth Hutchison headed goalwards and the outstanding Michael Bolochoweckyj cleared off the line.
Gordon was first to react and he swept in a close-range shot past the helpless Evans.
Ian Little almost added another against his old club right on time.