"We're going to hold a public display of our displeasure and send a loud and clear message that we won't tolerate racism regardless of who it comes from in our city," said Najee Ali, a civil rights activist.
Ali said hundreds are expected to the gather around 5:30 p.m. at the corner of 12th and Figueroa streets before Tuesday's playoff game between the Clippers and the Golden State Warriors.
Activists initially planned to call on fans to boycott the team, but have since decided they want Sterling to be banned from the National Basketball Assn.
"We want fans to support the team, especially at a critical time like this," Ali said. "Those guys, Chris Paul and Blake Griffin, they've done nothing to deserve this."
"We're not going to punish them, they have our support," he added.
Two websites posted a recording over the weekend that they identify as a conversation between Sterling and a woman named V. Stiviano. A person the websites identified as Sterling can be heard criticizing her for associating with black people — even though Sterling's team and the league it plays in are 80% African American.
The Clippers released a statement Saturday in which President Andy Roeser said the team does not know if the man recorded is Sterling, but that the comments do not reflect Sterling's "views, beliefs or feelings."
Furthermore, Roeser said the woman on the tape is the defendant in a lawsuit filed by the Sterling family that alleges she embezzled more than $1.8 million. Sterling told Roeser the woman said she would "get even" with Sterling for bringing the lawsuit against her.
In a lawsuit filed in Los Angeles County Superior Court on March 7, Rochelle H. Sterling, Donald Sterling's wife of more than 50 years, portrays a woman identified as V. Stiviano as a gold digger who seduces older, wealthy men and persuades them to shower her with gifts.
Sterling and Stiviano began an affair after meeting at the 2010 Super Bowl and were still in the relationship when the lawsuit was filed, according to the filing, which describes Rochelle Sterling as "a married woman seeking to protect and recover community property in her individual capacity."
On Monday, Leon Jenkins, president of the L.A. chapter of the National Assn. for the Advancement of Colored People, said it would no longer give Sterling a lifetime achievement award that Sterling was scheduled to receive next month and would also return an unspecified amount of money that Sterling donated.
Jenkins decline to say how much money Sterling had donated to the NAACP.
The NBA said it is conducting an investigation into the recording and plans to conclude its probe in the next few days.