Sherlock Holmes dismissed his staggering deductions as "elementary" when he dazzled the uncomplicated Watson. But his cunning seemed to place him excitingly on the edge of moral suspicion even though he was, in the end, always on the side of right.

As a child, I used to be delighted by the way Holmes reached his conclusions, partly because of his sheer intellectual panache but mainly, I think, because his methods seemed dangerous - almost evil - the product of a brain that, if it had ever turned to crime, would have given birth to acts so foul that even Moriarty would have blenched. Oh to be possessed of such powers!

Then one night on stage at the Edinburgh Fringe, when I was still working as a stand-up comedian, I talked to a woman in my audience. After a couple of exchanges - and thankfully some laughs - I just knew that the man sitting next to her was her second husband, for no reason that I can pinpoint.

Something about her language or posture or tone of voice - I don't know what exactly - had told me. When I mentioned it she looked as shocked as I felt at the fact that I knew I was right. It was my Sherlock moment. It indeed felt elementary to me and yet, in her startled eyes, I saw that she found it a little scary.

If you have a flash of insight like this by accident, you're a comic. If you do it systematically, you become what we call a mind reader. My own relationship with audiences has led me to develop something of a fascination with these performers. One of my favourites and one of the best, though not the best known, is the American Marc Salem, who has recently been performing in London.

So when I meet him in the lobby of a London hotel, it is with childish delight that I ask him to show me a trick. Salem takes my pad and writes something on the first page and then something on the next. He hands it back to me. He has written four numbers - one, two, three, four. "Choose a number from one to four". I choose three. "Turn over the page". It says "geniuses always choose three". I feel like I am now six years old and waiting for the birthday cake to arrive. Or I've just re-read Sir Arthur Conan Doyle's The Speckled Band for the umpteenth time.

And then I ask him how he did it. For when I say he is a mind reader, he's not a mind reader. None of the really impressive performers who do these tricks call themselves mind readers. No one can read minds, as we shall see. To say you can is a con. There are no pointy hats with Marc Salem or our UK version, Derren Brown. What they do is just pay very, very close attention to us and our behaviour for every second that we are in their company. And we in turn give them more clues than we ever dream we are doing.

Far from keeping our secrets from them as we think we are, we might as well be doing semaphore on a hill while they gaze at us through powerful binoculars, so easy do they find it to read us.

But however rational their methods, we love to think that what they do is spooky, supernatural, a defiance of the laws of nature. Yet at the same time we want to know how it's done. We know it's a trick, but we want to believe it's a miracle. We want to know the secret yet when we're told we are invariably just a little disappointed. Mind readers, for that is how we want to think of them, play with us so that we feel all those contradictions at once.

Welcome to a world that hovers between our faith in science and our desire for faith, suspended somewhere between Einstein, the Pope and Harry Potter.

So how did Salem know I would choose the number three? It's quite simple, although every time I have tried to do it since, I have got it wrong. Salem, though, rarely makes a mistake.

"I am hedging my bets", he says. "First I am guiding you linguistically. I have used three of the numbers when I said choose one to four. People tend not to go to the extremes and choose one or four. Some do but most don't. We tend to go down the middle. So we're down to two or three. If you're right handed you tend to go to the right. I've watched you taking notes. So I went for three." And then he says with a chuckle. "But the bottom line is if I am wrong, so what?"

Salem started to do this kind of thing when he was quite young. His father was a rabbi, his mother a pharmacist. "Both intellectuals and two of the funniest people I ever met," he says.

He thinks it started just with natural curiosity that developed until he was looking at things with a sharper understanding. He was "reading minds" at high school. He remembers once they moved house and his mum had a red hat she was looking for in one of the boxes. He knew which one. He says: "I suspect what occurred was that my mum on some level knew which box that hat was in and when I pointed to that one she stiffened or something not necessarily conscious and I just picked up on it."

He became fascinated with communication and what people pick up from each other and eventually turned that into a job as a researcher on children's television show Sesame Street. What the programme makers were interested in was how young minds worked. How did they best receive information? "It's about signs and signals, which is semiotics, without the pomposity of giving it a name and its own school of thought".

He started training a range of people - from FBI agents to company directors - how to read people. And now, because he has discovered the thrill of performing, he can make a living telling people their pin number, the surname of their best friend, the details of their wedding day or which word they have picked apparently at random from a book.

What he and Derren Brown do is harmless, fascinating and entertaining. Yet at the same time it seems to answer a need in us for characters in our world who know more about the universe than we do, who have access to powers that we don't. Both are at great pains to disavow any kind of supernatural element in their work.

It's not so much cosmic or elemental as, indeed, elementary my dear Watson. Brown calls the phrase mind reading "a lazy explanation", preferring to describe what he does as "exploring things through tangible psychological methods". Salem stresses that his show is "not occult, not supernatural. I have to do disclaimers because people want to believe in the impossible. They would love me to provide answers about themselves. But I just can't tell people who they're going to marry."

The literature of magic dates in earnest from the mid-16th century, as the exposure of witchcraft was linked to heresy and became a tool in the Church's armoury of coercion. Thousands of writings of the period cover everything from conjuring tricks and recipes for salves to "the literature of lowlife" as one text calls it. Magic in all the literature stretches from the supernatural - voodoo, witchcraft and evil spirits - to con artists and frauds. It is a family tree of deception and our credulity is engaged by each in different ways - from simple delight at a conjuring trick to the surrender of our lives to the cruel manipulations of cults.

All the writings have descriptions of what makes a great magician. For instance, Hocus Pocus Junior: The Anatomie of Legerdemain, published in 1634, suggests the following: "First, he must be one of an impudent and audacious spirit. Secondly, he must have a nimble and cleanly conveyance. Thirdly, he must have strange terms and emphatical words. Fourthly, such gestures of body as may lead away the spectator's eyes from a strict and diligent beholding of his manner of conveyance."

Chutzpah and charisma, fast fingers and abracadabra apart, the last is the one that best serves mind readers. When we are in a state of amazement, and they work us up gradually, our powers of observation and recall are at their most diminished. We cannot perceive what we have witnessed correctly. So we give them information in ways we don't imagine. And thus we are conned, though in the nicest possible way.

On Salem's showreel, he can be seen asking a woman to stop him as he flicks through the pages of a book, inviting her to choose a word off that page. He then "guesses" the word. He helpfully films her face from his point of view. "The word begins with A, B, C, D, E, F, G . . . F," he says, apparently reading her mind. And so he goes through the letters correctly telling her that she has chosen the word "fortune". But you'd have to be blind not to see the incredibly obvious blink she makes involuntarily every time Salem hits a correct letter. Thus he "reads" her mind. And she is duly astonished.

What he does rather is read her reactions. When Doris Stokes, the famous "medium", affected to know the name of your granny or the fact that you were Sagittarius, she was mostly either using stooges and information collected in advance or dealing in such generalities that it would be difficult not to answer her descriptions merely by being a mammal with a day job. Her act eventually became the object of jokes rather than awe. When I was a comic, we used to say that we all wanted to be the support act to Doris Stokes because at least if we died in the first half, we'd have someone to talk to in the second. But Brown and Salem use no stooges. In fact Salem has offered $100,000 to anyone who can show that he uses assistants or electronics. "They're not fair in this game", he says. Why? Because they are fake?

"No" he laughs, "Everything I do is fake! But I have nothing to do with spirits, nothing to do with mind transfer or whatever words these performers use. I don't spend a great deal of time analysing what such people do. Do I find some of the things they do surprising? Yes. Do I know where they get their info from? I don't know. But the ones I have seen certainly aren't talking to dead people. And the reason I say that is I have never heard them tell anyone anything that the person did not already know. The person always confirms what they are saying. They are not saying 'the money is in the basement under the big barrel' and you go and look and there it is!"

Brown and Salem are not, and do not claim to be, delving into the black arts. They occupy the ground where the science of observation meets the need for mystery. They are operating watchfulness together with what both acknowledge is a heightened degree of empathy. How we generate empathy remains a matter of enormous debate.

In 1996 three neuroscientists at the University of ­California, Los Angeles, conducted some experiments on a live monkey. Probing its brain they came across a cluster of cells in the premotor cortex, an area of the brain responsible for planning movement. Their key observation was that the cells fired not just when the monkey moved but also when it observed the same action performed by a human or another monkey.

They called these cells "mirror neurons" and began to develop a theory about the way that children develop the realisation that others have minds similar to their own. Children collect evidence of gestures and actions and use their understanding of that to explain other human beings around them. There are refinements of this and other theories but it is clear that empathy exists. What is more, there are techniques for enhancing it. A performer might for instance get you to tell a story that is significantly laden with a particular emotion. While you are recounting it, they touch you lightly on the arm. Then every time they touch you there again, you are likely to go back to that state of feeling.

And, as Salem says, we are all surprisingly alike - something that frequently rather irritates members of his audience. They are disappointed to find that they can be so easily read. But he says: "Believe you me, if you scare anyone anywhere in the world, their eyes are going to widen and their mouth is going to open."

His intuition about the clues we give, from this to the more subliminal ones that enable him and his fellow performers to tell you your pin number, is the articulation of a lifetime's experience. "There are physicians that can shake your hand and tell you what is wrong with you. They are not supernatural, they just have experience. A little ruddiness here, a little sweating over there, something in the eyes, the way your palm feels. It all adds up."

To extract specific information requires a deal of Hocus Pocus's fourth quality: suggestion. Mind readers create the conditions under which we give them information. "Magician's choice" it's often called in the business. It's the way he got me to choose the number three.

The truth is that we don't pay anything like enough attention to the non-verbal language that we all use every day of our lives. We only expect to be able to read people we know. We can finish our partner's sentence or know if a long-time work colleague agrees with us. We just don't expect a perfect stranger to be able to do this on stage at our first meeting. But Salem gives an eloquent example. Some man in the audience had told him that he'd had a trip involving some kind of grass or straw hut. "So I said (slightly obviously) 'Africa'. They said 'No'. And then I said The Globe [theatre]. And he fell off his seat. But the way he'd said 'no' was in a tone that said 'not even close, you're way off, think totally different'. And I happened to have been to the Globe theatre and its roof came into my mind."

We love being conned by magicians and mind readers. That's why we pay to see them gently dupe us. We are deeply engaged by flirting with the unknown but become rightly wary when the deception steps over the boundary of entertainment.

While even the most intelligent people read their horoscopes and believe them at least until the end of the paragraph or when the doorbell distracts them, the credulous can be drawn to the apparently supernatural with dreadful results.

Salem's most satisfying moment after a show was a conversation with a man who had brought his two daughters. They had both started to get involved with cults. "He wanted to prove to them that here was someone who can do the same things as these people said they could, without making any of the claims they were making. And the two girls saw it. He was a very smart man". As are the mind readers, who are Holmes to the cult leaders' Moriartys, and whose skill in the end is to make a habit of seeing the things that the rest of us just don't notice.

Simon Fanshawe is a writer, broadcaster and Perrier Award-winning comic. His most recent book is 'The Done Thing' (Century, £9.99)