The knock: Instead of training at a well-heeled thoroughbred track, Sherman had chosen to ready California Chrome at Los Alamitos Race Course.
"They thought it was like a quarter-horse track," Sherman said.
And they were right. But it didn't matter.
Sherman, back at Los Alamitos on Thursday, smiled and squinted in the sun. He sat at a table on a patio above the track where he watched the horses go by and shared laughs with friends, content knowing that California Chrome had proved the Churchill Downs folks wrong.
"Sure did," Sherman said of the horse that would win the Kentucky Derby and the Preakness.
But the confusion over Los Alamitos is understandable. The race course has, in fact, long been a quarter-horse track. It hadn't held a daytime thoroughbred race in more than two decades.
That changed Thursday when Los Alamitos opened a two-week thoroughbred meeting at its renovated course, betting that a bigger track will bring bigger crowds.
The track spent $1.5 million to expand its five-eighths-mile dirt oval to nearly a mile long. Its 1,380-foot stretch run is the longest of any North American race track. Another $5 million was spent on remodeled bathrooms, television monitors, an expanded winner's circle and infrastructure that will allow fans to watch from the infield.
Sherman called Los Alamitos "the up and coming place for racing."
The influx of capital has come at a time when racing in Southern California has shown signs of collapse. Hollywood Park in Inglewood shuttered in December and Fairplex Park in Pomona will close its doors later this year.
Those tracks' misfortune has become a boon for Los Alamitos. Since 1991, the track has stuck mostly with quarter horses — sprint specialists that don't typically draw the big names and big crowds of thoroughbreds. Now the track is gambling that thoroughbred racing can still be a viable business.
The attendance Thursday was 5,702 and the total take was more than $4.5 million — good if unspectacular figures. Officials had to draw new lines in the parking lot. Until now, the grandstand never filled up much, save for die-hards like James Mata, 45, of Fullerton.
Mata grew up across the street from the track and has been coming since he was 8. His grandmother took him to see quarter-horse favorites Dash For Cash, Gold Coast Express and Griswold. But even he sees the advantages of thoroughbred racing.
"You bend down to tie your shoe and the race is over," he said of the quarter-horse races. Moments earlier, he pounded his racing form against a railing as if using it to whip his horse, the race's eventual winner.
He high-fived a friend, as did others around him. There was excitement at the place where many used to come just to place bets at other tracks.
"When I come here on Friday nights, there wouldn't even be, for the quarter horses, I bet you there weren't even 300 people here," said Russell Wood, 57, of Whittier.
"Look, look!" he said with a laugh as he grabbed an intricate spider web that had festered on the seat in front of him. "Now they're gonna come probably clean this place because I think they're gonna make it here."
Already, Fairplex Park's thoroughbred dates will move to Los Alamitos, and the track will still run quarter horses. And it is still home to the sport's new star, California Chrome, who has his stable at the track. On Thursday, marquees outside the track proudly displayed the horse's success, and California Chrome T-shirts were the opening day giveaway.
For some fans, that was reason enough to attend. Tom Whitby, 83, spent about half a century working around horses, including at Los Alamitos, for his brother, a trainer.
"I just came out for the shirt," he said from the grandstand, taking a break from peering through his binoculars.
Nearby, Wood watched the horses race down the vast stretch run as cigar smoke curled up past a ponytail of an equally smoky hue. He only smokes for luck, he said, and his luck was bad.
"I got No. 5 in this one," he said, referring to a horse that dawdled far behind the next slowest thoroughbred. "And there he comes in last again!" Wood added, smiling big. "If they could let me bet in reverse, I'd be a rich man."
Wood bets only small amounts — for fun. He said he has always been a people person. With the renovations, he hopes he'll finally have lots of people to commiserate with.