The moment the icy splinter of fear entered my heart, four years ago, occurred, as it has for so many of us, at a dinner table. “Don’t you think that Israel is becoming very unpleasant?” said one deep thinker. “We used to be on their side because they were the underdog, but now they’re so aggressive.” That was not the moment. It was the next comment, made as I was taking a deep breath, by another guest. “Well,” he said, “I never thought about that before, but, yes, you’re right.”

That person was someone who had for several years been a good friend, good not only to me but in general. He is a kind, compassionate man, quick to offer practical help and moral support to his friends. He does a great deal of unpaid work for charity. His words took me back to a time when the same sort of mindless verbal ping-pong was played over other tables, when Gentiles in England dismissed reports from central Europe as hysteria or propaganda. I later said to my friend, who never reads a newspaper, that he shouldn’t comment on topics he didn’t understand. He protested that he wasn’t commenting: he was “just agreeing”.

In New York, where I grew up, I never heard remarks of this type, not simply because of the number of Jews living there, but because my accent and appearance identified me immediately as one of them. Since moving to Britain 20 years ago, I have learned that others see me only as an American or a New Yorker. I therefore came late to the sort of disconcerting encounter that European Jews probably take for granted – the person of respectable and benevolent appearance who, chatting to us in a railway carriage or a coffee shop, hopes we do realise that the Jews are plotting to steal our gold and rule the world.

That type of person – as well as those, of course, who won’t hire Jews and those who vandalise synagogues and cemeteries – is what most people think of as an anti-Semite. But I would suggest that the definition be made broader to include those who let unpleasant remarks about Jews go unchallenged, who don’t consider the subject to be worth a fuss. Those, in other words, who feel that we are not worth defending from the mindless vilification that has been increasing over the past several years, and zooming up since the air strikes on Gaza.

It was not the first time my friend had startled me with a remark of this kind. We had met not long before 11 September 2001. About a week after the World Trade Center was destroyed, he said to me, “I don’t mean to offend you by saying this: I just wonder if you think this could be true. Someone told me there was a rumour that the Israelis were responsible.”

That remark, however, passed me by in an I-didn’t-hear-that moment because I was already reeling from the reactions of the “America deserved it” crowd, and couldn’t take in anything more. But later I reflected that there was a point at which innocence and ignorance are not the same. As recently as 50 years ago, it was normal to think that homosexuals sought to corrupt pure young boys, and that children who said an uncle or priest had touched their private parts were dirty little liars. Nowadays, anyone who

espoused such beliefs would be ridiculed and might be up on a charge – as would someone who believed, as people did 700 years ago, when the Jews were expelled from England, that we kill Christian children and use their blood to make matzos.

My friend and I remained on good terms until last year, when he asked if I would join him on a trip he was very eager to take – to Syria. As my heart sank deeper and deeper, he enthusiastically described the archaeological treasures, the history, the romance.

“I know all about those,” I said sadly, “but do you know that Syria is a hotbed of anti-Semitic terrorism? Their newspapers and radio and TV are full of attacks on Jews, and some of them actually say it is part of our religion to kill babies.”

He was silent for a moment, and then sighed. “Oh, can’t you forget about that? Just for two weeks?” I said I couldn’t.

My friend departed alone for Syria – where, he told me, he had a marvellous time and didn’t hear a single anti-Semitic remark – and I was forced to conclude that, sadly, as we say in my native land, three strikes and you’re out.

I never thought I would end a friendship on political grounds and it distressed me greatly to end this one. Was I simply taking personal offence at my friend’s unwillingness to treat me with imagination and sympathy? In the end I decided that if the comments had insulted only me, I could have chosen to ignore them, but, as they did not, I could not.

Many non-Jews will probably think that I behaved in an intolerably pompous way towards someone who has no political influence whatsoever, and that I am elevating a personal slight to an absurd degree. But, as the last sentence of my favourite novel, Middlemarch, says, “the growing good of the world is partly dependent on unhistoric acts”.

Surely that goes for the growing evil, too? How do we know what effect a word or a nod may have? A single remark has at times been enough to alter a life. And not only words but thoughts – recorded in surveys or the ballot box – sway politicians to pass, execute, or ignore our laws.

Unfortunately, many of the sentiments that make the English so agreeable – their diffidence, tolerance, witty detachment – mean that “nice” anti-Semitism is practically bred in the bone. It’s a condition that has less in common with bomb-throwing than with the reluctance of most people to tell off those who are discourteous or disgusting in public, with the results we know. Just as the “nice” passengers on the bus or train turn themselves into zombies when others start shouting obscenities, the nice guests at the dinner party pretend, at an awkward moment, that they have heard nothing amiss.

Another friend was at such a party when a turn in the conversation made it relevant for her to say she was Jewish.

A man asked, “You’re Israeli?”

“No,” she said, “I’m English.”

When he asked her to explain this apparent paradox, she said that she and her parents had been born here. “But,” her interlocutor continued, struggling with this concept, “you’re different from the rest of us.”

When no one else said anything, my friend decided it was time to leave. She knew that there was no point in challenging the company, or taking up the matter with the hostess later, because, like me, she had done so in her youth and met only embarrassment and resentment. Why, we were asked each time, did we (and not the person who had made the remark) have to create unpleasantness?

I can understand the reluctance to turn an amusing evening into a trial for thought crime. But the riposte to bigots need not be solemn and drawn-out. Once my fork stopped halfway to my mouth when a film director’s bimbo girlfriend came out with an offensive characterisation of Jews, which she followed with the defence: “But I’m not anti-Semitic. No one can say that I’m anti-Semitic.”

A Gentile screenwriter replied: “I’m afraid, dear, I’ll be the judge of that,” and got a laugh. I then got a bigger laugh with, “No, I’ll be the judge of that,” and we moved on.

People may sometimes be deterred from objecting to remarks about Jews because they don’t feel qualified to judge whether the comments are true. But it is not necessary to know a raft of facts before you challenge a statement. You need only refuse to accept it unquestioningly. Be suspicious, I would urge you, of statistics that are presented as carrying intrinsic moral weight. (You hear a flagrant example of the current rush to judgement from people who want to arouse horror by pointing out how much higher the Palestinian casualty toll is than the Israeli. In response to their “That’s so not fair!” one might mention that German deaths in the Second World War were many more than deaths of US and UK forces and civilians. Should one, therefore . . .) And anyone announcing, with sanctimonious condescension, that the Jews today are just like the Nazis of yesterday should be handed a history book and asked: “Do Israelis do this?”

I should not like to leave the impression that my life in England has been characterised by anti-Semitic prejudice and hatred. Far from it; I have enjoyed prosperity and pleasure here, and my Jewish friends would say the same. But the intelligence and good manners we have known so much of the time make even more shocking those moments when they fall away. We wish that more of you would speak up when you hear ignorance and nastiness paraded, and not remain silent, like spectators to a crime, though the crime in this case hasn’t happened. Not yet.

Rhoda Koenig is a former nightclub singer, travel writer, literary editor and theatre critic for Punch and the Independent