Nothing Ventured : CAST IN DOUBT, <i> By Lynne Tillman (Poseidon Press: $20; 239 pp.)</i>

Flaubert once expressed the desire to write a book about nothing at all. Such a book, were it ever written, would not necessarily lack sense, or structure, or even plot and character, but would contrive its meaning out of its own innards. Like a maze or a labyrinth--or a fiction by Borges--this book about nothing would be a self-enclosed system, whose various elements derived their significance not from their resemblance to reality but from their relationship to one another.

Not least of the strengths of Lynne Tillman’s new novel “Cast in Doubt” is that it delivers the goods, in terms of traditional narrative pleasure, while at the same time undertaking an investigation into the nature of fiction. So subtly and elegantly does Tillman phrase the questions, the reader may emerge wondering if all books aren’t in the end about nothing at all.

“Cast in Doubt” is set in Crete in the 1970s and is narrated by a 60ish homosexual American writer named Horace. Expatriate communities on Mediterranean islands tend to be bitchy, defensive, incestuous and self-important, and Horace’s is no exception. Tillman has caught the social tone beautifully. The queen bee is Alicia, ex-opera singer, stately from yoga and omniscient about island affairs. Roger is the promising writer who came to Crete to finish his second novel and years later is still at it. There’s an eccentric Englishman named Stephen the Hermit, devastating in his youth but now quite bonkers. These and others spend warm evenings in terrace restaurants getting drunk and being spiteful and arguing about Gertrude Stein and Ezra Pound. The tremors from a larger world--Nixon, Carter, the CIA--are registered, but faintly.

This is Horace’s world, and he is reasonably content with it. Horace has come to terms with things. He makes an adequate living writing detective novels, runs a moderately satisfying relationship with a Greek youth named Yannis, and has long since settled on a few simple pleasures. He is fond of fish, for example, and very fond of white wine. He sometimes makes a fool of himself at the end of an evening, and the literary and sexual vendetta he has sustained for years with the appealing Roger clearly stimulates and amuses both men.

Horace, however, lives more in his mind than he does in the world. He reflects unceasingly on all matter of topics: homosexuality, aging, history, art, other people, writing, life. Horace is rarely bored and never boring. But what engages his attention most strongly as “Cast in Doubt” opens is the recent arrival in town of a young American woman with dyed hair and a ring in her nose. This is Helen.

Horace is obsessed with Helen. She comes from Manhattan, as he does, yet he cannot understand her. This is partly of course because Helen is 40 years younger than he. He watches her on her balcony in a manner that he admits is voyeuristic. He develops a prurient interest in her sexuality. He ponders the nature and causes of her unhappiness. Then it emerges, after a young man cuts his throat because she apparently rejected him, that Helen may have had a sister who may have been a twin who may have attempted or even succeeded in killing herself.

The mystery that surrounds Helen (at least in Horace’s mind) appears to deepen here, and for a while there’s a whiff of Patricia Highsmith. But Horace’s dark musings are curiously self-conscious. “I am often a trifle bored,” he writes, “and I try, in my own way, to make life a bit more interesting, more inventive, more like fiction than it might otherwise be.”

While Horace is secretly at work on a serious novel called “Household Gods,” he frankly enjoys the mysteries he “tosses off”--and which pay the bills--and he reflects at length on the methods of his detective, a character named Stan Green. Horace, as he ponders the enigmatic Helen, seems to be applying to his own life the techniques of his fictional detective, and he is not blind to the risks involved. “I am,” he admits, “utterly susceptible to intrigue, my own and others.”

The novel at this point begins properly to articulate the mystery at its heart, and that mystery is whether in fact there is a mystery at all. For given Horace’s tendency, as a writer of mysteries, automatically to interpret life in the same terms, the reader--indeed Horace himself--now suspects that the sinister plot he is constructing from a few suggestive clues may have no existence in reality. But then, quite suddenly, Helen disappears. The dark and eerie spectacle that Horace discovers in the room she’s left behind acts as the spur that drives the quest that moves the book to its climax and resolution.

In her previous fiction, Lynne Tillman’s attitude to conventional narrative structure has been uneasy if not outright critical. “Haunted House,” her first novel, was composed of three alternating strands, the accounts of three young women who never meet, whose lives are not interwoven in any way. What connections there are occur at the level of the ideas and images they each employ as they try to make sense of their lives. Then, in “Motion Sickness,” a woman drifts across Europe, from hotel to hotel, her experience a journey but one that lacks the sort of accelerating thrust to a destination that generally characterizes fictional journeys. Again, the story, the design, the thematic interconnections, all occur at the level of the narrator’s reflections upon the somewhat random events and people she encounters en route. In both these books, distinctively fresh narrative forms are proposed for the contemporary novel.

In “Cast in Doubt,” Tillman employs the narrative techniques she so deliberately eschewed in her earlier work. She employs them with a high degree of self-consciousness and in a broadly ironic tone. She inspects them, these hoary devices that have hooked and held readers since Homer; she reflects upon their provenance and shakes them vigorously to see if there’s anything inside. She also depicts with color and brio a Mediterranean way of life that for all its insularity is extremely attractive; tells a story that might not be a story but even if it isn’t ends up being one anyway; and most especially, paints a portrait of a mind, Horace’s mind, that is broad, funny, self-deprecating, blind in spots, occasionally ridiculous, capable of passion, and relentlessly entertaining.

Horace, we learn in a postscript, goes on to write two more detective novels, one called “The Big Nothing.” It is a paradox of fiction that while in a sense all novels can be seen as big nothings, being big nothings doesn’t prevent them from being at the same time witty and perceptive about the ways our minds work in their unceasing attempts to make sense of the world, fashion order out of chaos. In that sense we are all novelists, we are all Horace, with Horace’s tendency to imagine what we cannot know, embellish what we do know, and organize it all in a way that satisfies us, entertains us and allows us to keep going. Truth is not the point. The point is to keep telling the stories, even if they’re cast in doubt.