The Swiss-born artist Paul Klee (1879-1940) painted with his left hand and wrote with his right and with both hands could draw and write simultaneously, “taking his line for a walk”, as he put it. His paintings often depicted the world as he believed the modern artist should, in a language of lines, signs, symbols, formulas and rhythms that went to the deeper meaning of things. Logical yet irrational, intellectual yet mystical, apparently childlike yet carefully constructed, often slyly humorous, many of his paintings have inspired composers, among them Pierre Boulez, who wrote a book about Klee in 1989, The Fertile Country (named after Klee’s painting of the fields around the Nile).

Klee was an accomplished semi-professional musician, born into a musical family; his German father was a music teacher, his Swiss mother a singer. Until the age of 19, when he opted for art and moved to Munich to study painting, he envisaged making the violin his career. After a long courtship, he married a pianist, Lily Stumpf. His other link to the theatre is more psychological: Klee was a wry observer of humankind; his diaries reflect the ironic distance with which he studied friends, strangers and himself. He was known to take his opera glasses on walks so as to observe people on the sly.

Summoned in 1920 by the architect Walter Gropius, Klee taught for 12 years at the experimental Bauhaus school, sharing a house at one point with Kandinsky. Those were halcyon years, but things changed with the rise of Nazism; he moved back to Switzerland in 1933, aged 54, having been fired from the Düsseldorf Academy. In 1937, 17 of his works were exhibited in the Degenerate Art show in Munich, and 102 were confiscated from public collections and sold. Yet in spite of feeling intensely demoralised, and in spite of the ravages of a skin disease that left him virtually unable to swallow, he entered his most productive years, producing larger, anguished work in rough, broad brushstrokes. He died in Bern seven years later.

This show, Paul Klee: Theatre Here, There and Everywhere , opened at Bern’s three-year-old Paul Klee Centre last year and has been extended for the Palais des Beaux Arts with a new section curated by the 84-year-old Boulez. The show’s theatrical theme gives coherence to the selection from among a vast oeuvre. Most of the 200 watercolours and drawings have been lent by the Bern museum (it has 4,000 pieces), with a few coveted loans from New York’s Museum of Modern Art, Paris’s Centre Pompidou, Philadelphia’s Museum of Modern Art and Washington’s Phillips Collection.

Christine Hopfengart, the show’s Swiss co-curator, says that putting on a show by Klee outside Bern is a logistical nightmare because his works – very dry watercolours often painted on cheap material, oils mixed with watercolours on newspaper – are very fragile. She points to his 1939 “Leviathan” in particular, an almost Art Brut portrait on wrapping paper irregularly glued on to cardboard.

Klee’s works are not easy to decipher. A day’s walk in the countryside ending in a storm, Klee wrote, might be recalled via stains, dots, surfaces plain, flecked or striped, undulating movements and movements interrupted or articulated, each corresponding to a sensation or a memory.

In the non-musical section of the Brussels show, most of the works are representational and thus easier to read. Each of the rooms, dimly lit to protect the works, concentrates on disciplines that Klee amply reflected in his work: theatre, dance, circus, masks and puppets. On one side are photographs, programmes, tickets, letters, even old films; on the other Klee’s drawings and paintings. The 1923 “Tightrope Walker” gingerly walking across a horizontal line hanging over a perspective that evokes a face – a twirl for an eye and a dash for a mouth – also conveys Klee’s artistic preoccupation with equilibrium. The theatre is used as a metaphor for the human condition – disarticulated puppets, teetering acrobats, broken masks, tragic clowns, posturing singers.

The music section is, predictably, more abstract and more exciting. Boulez discovered Klee at a show in Avignon in 1947; two paintings in particular, he felt, evoked the alphabet of an unknown language. In one room, a Yamaha Disklavier grand piano plays 12 of Boulez’s pieces, which probably wouldn’t have been to Klee’s taste. He didn’t enjoy Stockhausen or Hindemith, whom he knew personally, far preferring Bach and Mozart, who, he said, taught him more about painting than any great painter. His first powerful musical experience was a performance in Bern of Offenbach’s The Tales of Hoffmann, which seduced him by its mix of earthly and ethereal spirit.

The 60 works Boulez has selected show Klee experimenting with line, chequered patterns, dots and symbols; on the walls, extracts from Boulez’s book evoke some of the musical parallels he found in these paintings. Two paintings hang side by side: “Monument to the Fertile Country”, an overhead view of Egyptian fields forming a pattern of rectangles and mastabas, which Boulez compares to a fugue; next to it, “Fugue in Red”, with its echoing images of triangles, circles, leaves and amphoras, is not really a fugue but a canon, says Boulez. But Boulez’s interest is far from literal, unlike the room of contemporary videos of grimacing faces and clapping audiences that strikes the show’s only false note.