South Africa chased down the target of 240 runs on a rain-affected day 4.
Dean Elgar played a match-winning and unbeaten knock of 96 runs to get his side over the line.
The series is now level with 1-1 and the next match at Cape Town will be the series decider.
India 1st innings 202 all-out
Rahul 50 Jensen 4-31 Ashwin 46 Olivier 3-64 Agarwal 26 Rabada 3-64
South Africa 1st innings 229 all-out
Petersen 62 Thakur 7-61 Bavuma 51 Shami 2-52 D. Elgar 28
India 2nd innings 266 all-out
Rahane 58 Ngidi 3-43 Pujara 53 Jensen 3-67 Vihari 40* Rabada 3-77
South Africa 2nd innings 243-3
Elgar 96* Ashwin 1-26 der Dussen 40 Thakur 1-47 Markram 31 Shami 1-55
Day 4 was badly affected by rain and no play was possible in the first two sessions.
The ground staff did a wonderful job as the outfield got dry and the match started in the third session.
A minimum of 34 overs were allowed to be bowled on day 4 but the South African batters took only 27.3 overs to chase down the remaining 122 runs and register a historic victory over India.
This was South Africa’s first-ever victory against India at the Wanderers stadium in Johannesburg.
It was also the highest successful chase by South Africa against India.
Elgar’s brilliant knock
The South African skipper Dean Elgar played an excellent knock today as he reminded unbeaten on 96 runs.
He led his side over the victory line today against India.
Dean Elgar also scored 86 not out the last time when South Africa lost against India at this ground while chasing 241 runs but today he scored the winning runs for his team.
Dean Elgar and Rassie van der Dussen started the day positively and both of them played aggressive cricket.
Bumrah started with an almost unplayable delivery to van der Dussen but he survived that delivery.
Ashwin started the bowling from the other end in the start but after bowling only two overs he was replaced by Shami.
Rassie played some brilliant shots on the bowling of M. Shami.
He smashed some superb boundaries against him.
Dean Elgar also scored quick runs today to finish the game as soon as possible.
Shami gets breakthrough
M. Shami broke the brilliant partnership of 82 runs between these two by getting van der Dussen caught out at the slips.
Shami bowled outside off-length delivery that was moving away, Rassie got an outside edge that went straight into the hands of Pujara at first slip.
Rassie van der Dussen scored a crucial knock of 40 runs.
Bavuma and Elgar built up another important partnership and both of them remained unbeaten till the end.
Both Bavuma and Elgar played wonderful shots all around the ground today.
South Africa chased down the target of 239 runs with ease in the 68th over to register their first win against India in Johannesburg.
Shami, Thakur, and Ashwin picked up 1 wicket each.