Longtime USC assistant coach Bob Cantu will serve as interim head coach while a search for O'Neill's successor is conducted.
"I would like to thank Kevin O'Neill for his four years of service to USC," Athletic Director Pat Haden said. "He took over a program under difficult circumstances because of our NCAA issues and, despite that, he was able to do some good things.
"It was hard for me to evaluate him as a head coach until this year when he had enough players and veterans to compete. As the season progressed, it became evident to me that we needed new leadership in our men's basketball program. Despite a nice road win in our last game, I felt it was best to make a change now, with most of the Pac-12 season still ahead of us, in order to re-energize our team.
"I feel confident that Bob Cantu will capably guide our team on an interim basis while we search for a new coach."
O'Neill was in his fourth year at USC and had a 48-65 overall record, including 7-10 this season.
"I enjoyed my four years at USC," O'Neill said. "It is a special place. I enjoyed the people I worked with, the players I coached and our fans. I thank USC for allowing me to guide their program. I couldn't be more proud to work anywhere.
"I would encourage all of our recruits to honor their commitment to USC and help build the Trojan program in the right direction."