The 20-year-old from Canberra ended Federer's hopes of a fourth title at the Spanish tournament by rallying for a closely fought 6-7 (2), 7-6 (5), 7-6 (12) victory.
Federer said his problem was that “I couldn't return his first serve.” Federer described his game as “just a horrible performance on the return.”
Kyrgios said he had been excited at the prospect of playing his childhood hero.
“Last night it was a bit tough to get some sleep knowing that I was going out there,” he said. “I've been playing well recently on the clay, so I knew I had a good chance to go out there and do well. I stuck to my game and served well and I got the win.”
Earlier, Rafael Nadal broke Steve Johnson's serve three times and advanced to the third round by defeating the American, 6-4, 6-3.
Trying to serve out the match at 6-5 in the deciding set, Azarenka double-faulted on the final three points to gift Williams the game.
Williams then dominated the tiebreaker and improved her head-to-head record against Azarenka to 15-3. Azarenka is 0-9 against top-ranked players.
“She almost hit an ace. She missed it by not even three centimeters. I thought, `Wow,”’ Williams said of her rival's serve at 40-0. “This is definitely an escape. I've had some other escapes, but I don't think I've been down love-40 on someone's serve before.”
After an injury-affected season last year, Azarenka, a two-time finalist, was unseeded in Madrid for the first time.
“She's absolutely on her way back,” Williams said of Azarenka's return to form. “It's so good to see a Grand Slam champion like her playing so well.”
Defending champion Maria Sharapova advanced to the quarterfinals by beating Caroline Garcia of France, 6-2, 4-6, 7-5.
The third-seeded Russian, who is making her sixth appearance at the Madrid tournament, broke her opponent's serve five times but was broken three times.
“It came down to a few points in the third set,” Sharapova said. “She was going for it. I could have had a higher percentage of first serves in, but even on good second serves she stepped in and hit really good shots and there was not much I could do until she started making a few errors.”
Also, in the woman's half of the championship, fourth-seeded Petra Kvitova of the Czech Republic defeated Anastasia Pavlyuchenkova, 6-1, 6-4 and fifth-seeded Caroline Wozniacki of Denmark beat Agnieszka Radwanska, 6-3, 6-2.
Third-seeded Nadal extended his record of wins on clay against American opposition to 10-0 by beating the 25-year-old from Redondo Beach, California in one hour, 14 minutes.
The Spaniard is aiming to extend his streak of winning at least one European ATP World Tour clay court title in the run-up to Roland Garros to 11 consecutive years.
Sixth-seeded Tomas Berdych of the Czech Republic also booked his place in the third round by beating France's Richard Gasquet, 7-6 (3), 7-5, while seventh-seeded David Ferrer defeated Albert Ramos-Vinolas, 6-4, 6-0.
The 10th-seeded Grigor Dimitrov of Bulgaria rallied for a 3-6, 6-2, 7-5 victory over Fabio Fognini, John Isner of the United States beat Thomaz Bellucci, 7-6 (5), 6-7 (11), 6-1 and Simone Bolelli defeated Luca Vanni, 4-6, 6-3, 6-2 in an all-Italian matchup.
France's Jo-Wilfried Tsonga defeated Jack Sock of the United States, 6-3, 1-6, 7-6 (4).