Swedish provocateur Ruben Östlund looks at selfless, and selfish, behavior in Cannes prizewinner ‘The Square’

Shortly before he began shooting “The Square,” Swedish director Ruben Östlund sent one of his trademark found YouTube videos‎ to the film’s cast.

Östlund loves putting characters through uncomfortable ringers to see what happens — a kind of artistic version of a cardiac stress test — and was fascinated by the live clip. In the 2006 video, Guy Goma, a Congo-born man who had come to interview for a lower-level IT job at the BBC, was mistaken for a famous Internet music expert. For about 120 cringe-inducing on-air seconds, Goma fought earnestly to answer questions he clearly knew nothing about.

“What I love about the clip is it points out how as human beings we’re always role-playing,” Östlund said. “How in order to interact to each other, in order to avoid chaos, we adapt to those roles, even if they’re roles we’re not good at playing.”‎

What seems bad to those under the lens could be a beautiful lesson for those peering through the microscope. ‎‎So argues Östlund, at least. Born to a sociologist mother, the director has long been interested in what folks really feel and want beneath interpersonal niceties‎ — witness the marital and martyr politics of his previous work, 2014’s “Force Majeure,” which made Oscar’s foreign-language shortlist.

With his new movie, Östlund has made what amounts to a kind of breakthrough in cinema-as-social-experiment. “The Square,” which opens Oct. 27, thrusts its characters into a slew of situations that are conventionally dramatic, sure. But what the workaday scenes really aim to do is pull back the curtain on the forces, more thinly veneered than we realize, that impel the human animal.‎‎

As the 43-year-old director and his cast gathered for lunch at a restaurant at the Toronto International Film Festival last month, they hashed out why — and how — they had embarked on such an unusual cinematic adventure.

Recent Emmy winner Elisabeth Moss, Danish veteran actor Claes‎ Bang, movement specialist Terry Notary — they all wanted to understand, well, just what Östlund was trying to understand.

“He’s holding up a mirror,” Bang said. “Not to society or the world, but how we see ourselves — like, as do-gooding people, are we really just doing things to feel good about ourselves?” Bang paused. “I think that’s what he’s doing.”

Ostensibly about the ordinary struggles of slick art-museum curator Christian (Bang), the English- and Swedish-language “Square” puts its denizens through the psycho-social paces.‎

There’s the complex set of reactions Christian feels after he first saves a woman from an attack and then realizes he was the victim of a wallet-stealing con, self-congratulation curdling into indignation.

There are the artists who set up a viral-video provocation that goes too far.‎

There are the numerous scenes with class overtones as Christian ignores, then feels remorse, about those less privileged around him — he first overlooks then throws money at a homeless person outside his local sandwich shop.

And there’s one-night stand he has with an American journalist (Moss) who asks him for the used condom that he suspects — correctly or arrogantly — will be deployed for other purposes.

“You genuinely don’t know where a scene is going to go,” Moss said. “But there’s so much commonality in the scenes. Like, you leave your wallet somewhere and trust no one will take it. But you take out a credit card just in case. Ruben has a way of highlighting how similar we all are, how flawed we all are.”

Östlund chimed in: “We have the ability to do good or bad. I need to understand that behavior. I don’t want it to be about guilt or judgment but the position we’re in, and how context is affecting us. We always have to put ourselves in context.”

The lessons of “The Square” so struck jurors at the Cannes Film Festival that they gave it the top prize, the Palme d’Or — a‎‎n upset, given both the director’s moral pugilism and his junior status in the seniority-minded galaxy of international cinema, equivalent to a group of under-18 Swedes winning the World Cup.

So surprising was the victory that Östlund and the cast lost track of the trophy.

“We almost forgot it when were on this photo shoot in Stockholm,” Östlund recalled.

“And it’s very expensive — it has all this bling on it,” Bang added.

“We were in the cab and it was ‘Do you have it? Do I have it?’” Östlund said.

That kind of wry iconoclasm marks Östlund, an artist at once of great anthropological seriousness and deep, class-clown silliness. The director has a snaggle-toothed comedic spirit that suggests that if he weren’t as fierce an intelligence or artistic talent, he’d be playing some clever practical jokes in whatever Gothenburg office park he’d be working in.

Also, he’s from Gothenburg but made his movie about Stockholm, which is a little like a San Franciscan getting their revenge by poking at the foibles of Los Angeles.

Östlund is a director who gives Moss’ character a monkey for a roommate — it wanders through her apartment unremarked upon — just to shake things up.

At one point in the Toronto lunch, a stranger came up to the table and, saying nothing, poured water in an embarrassing area on Notary’s clothes. It turned out to be a friend of the actor trying to get his attention. “That is great,” Östlund said, marveling at the gesture several times. “I’m totally stealing that for a movie.”

Notary powers the film’s pièce de résistance, a scene at a black-tie dinner in which patrons sit idly by as a piece of performance art goes problematically way over the line. To describe the scene is to rob it of its visceral power, but suffice to say that its concern is how humans move in packs — and how even (or especially) the most enlightened act with cowardice.

Notary, known for simian movements in works such as “Kong” and “War for the Planet of the Apes,” had to perform like an unleashed animal amid members of the twee-and-tony set, pushing them to the brink.

To help him prepare, Östlund showed Notary the scene from “Barry Lyndon” in which a boy disrupts a recital in oversize heels as well as clips from the infamous punk rocker GG Allin, who was known for engaging in radical acts such as attacking audience members.

“People ask if my intention is to create an uncomfortable feeling [among moviegoers], and the answer is yes,” Östlund said. “‘Awkward’ is the word most googled on the internet. I think we like it because we get an adrenaline rush. We can look and it doesn’t have to be us.”

Notary said he thinks the black-tie scene has a weirdly unifying effect. “Without browbeating the audience, it opens them up, and at the same time it puts all of us under one common denominator of ‘we’re kind of ridiculous,’” he said. He then described one actress’ startled response to his movements during the shooting of the scene.

“I think she fell in love with you,” Östlund said.

But it is perhaps leading man Bang who is truly the perfect vehicle for Östlund’s message. The actor brings his patrician bearing and good looks to Christian’s shiny and self-satisfied unawareness, which the director argues characterizes our collective moral self.

“There are points [shooting a scene] where you don’t know where it’s going to go, where you don’t know what the point of a moment or a reaction is,” Bang said. “But you know Ruben has figured it out and you trust that.”

“I’m happy that you think so,” Östlund replied. “I’m terrified half the time we won’t be able to solve it.”

See the most-read stories in Entertainment this hour »

calendar@latimes.com