As the vicar dipped his finger in the ash of last year’s palm crosses on Wednesday evening, drew a new one on my forehead and invited me to remember that I am dust and will return to dust, I knew immediately what I was giving up for Lent this year. Atheism.

It won’t be hard because my atheism has waned in recent years anyway. I gave up not going to church some time ago. Most Sundays I am there, praying and singing — another lapsed atheist hoping that the non-existent God he was brought up not to believe in doesn’t see.

Although in my case, that was the Jewish God (same guy, different PR): brooding, vengeful, unforgiving, speaks English only as a third language and demands Sabbath observance on completely the wrong day.

My parents didn’t believe in him either, so I was ritually circumcised on the eighth day after my birth to satisfy the grandparents and then left hanging. No Hebrew classes, no Jewish environment, no bar mitzvah. My mother tells me now that I didn’t want one. And that’s fine. I wouldn’t want to argue with me either.

But it left a hole. One that I filled at boarding school with the daily service in Westminster Abbey, weekly Latin prayers (pater noster qui es in caelis, sanctificetur nomen tuum) and compline in St Faith’s chapel every Friday night with the chaplain, Willy Booth. Nobody went except my mate Bob and me. Whether I believed a god was listening or not didn’t seem germane. Without us, Willy would have been doing the verses and responses on his own, sniggered at by the bones of Chaucer and Edward the Confessor. And we couldn’t have that.

In later life it didn’t come up. Atheism is the assumed default position of every modern urban adult. Although lately, many atheists think we want to hear their irrefutable arguments against belief and witty putdowns of the faithful, but I just wonder, “Why do you bother? To whom are you talking? Who do you think is not already an atheist?”

I think possibly these people grew up very close to religion and believe it is important to be endlessly forsaking it. But that isn’t the case with me. My childhood was godless, and there was room for improvement.

My father’s death changed things a bit. It seemed to me, at the very end, that God might have been useful to him. And it would certainly have helped us, when we buried him, to have had some formal tradition for the ceremony, rather than having to make it up as we went along, finding an irreligious rabbi prepared to conduct a partially Hebrew service amid crosses and carved angels with music from a cantor drummed out of the United Synagogue for being gay.

I found myself in shul a few times after that, representing my side of the family at what we call “simchas” (celebrations), but always felt awkward: unable to take part at any level, naked in my apostasy, uncomfortable in the yarmulke, bullied into wearing borrowed prayer shawls that made me feel fraudulent, understanding not a word of the Hebrew, freaked out by the separation of the women, my children regarded as mongrels …

Reform Jews tell me I should come to them but, alas, the kind of Jew that I am not is Orthodox, and I cannot help but look down upon the Reform, with their mixed congregations and English prayers. It’s like the old gag about the Jew stranded on a desert island who builds two shuls: one to worship and one to not set foot inside.

Which is where the Church of England comes in. That’s my language in that prayer book, my tradition, my education, my country, my poetry. And there is a building for it, usually a pretty one, on every street corner, opposite the pub, and any Englishman or woman can go in (to either) and take succour.

That is why, when Esther and I walked into St Bride’s in Fleet Street 15 years ago next month, the canon married us without a quibble. It’s what an established church is for. I wouldn’t have felt properly married anywhere else.

And when, some years later, my son, brought up like me into no tradition at all, said he wanted to go to church, I said OK, and we walked up the road to our local one the following Sunday and went in. And we’ve been going ever since.

Inside, it is vast, awesomely rectilinear and full of light from the high windows that are gently stained but only in squares, not pictures. The walls are white plaster and a little distressed, like a natural wine bar in Clapton. The congregation is small but intense. The vicar is young, well-read, quick-witted, unpompous and endlessly available to Sam to answer arcane questions about church procedure and biblical history.

The homily is always good, the organ music magical. There are bells and incense, bowing and genuflecting, a sung Eucharist and much talk of saints and the Virgin, which I think make this what you lot call “high church”? Anglo-Catholic, is that? Oxford Movement? I should know, I went to Keble College. But I don’t. I was brought up not a Jew, as I said, rather than not a Christian, so the distinctions are opaque to me.

And I do not not believe. I am not without faith. It’s weird, because Judaism does not require faith, only observance. Christianity is the other way round (right?) So I observe, like a Jew, the Christian service. And I have a sense that God is there — in the tradition, the words, the 2,000 years of conviction, the imagination of all the people who came before me — that I don’t get, I’m sorry, in a synagogue. Or, like, Pizza Express.

Sam on the other hand is filled with the Holy Spirit like a Baptist. When it comes time to kneel, he prostrates himself entirely. Which is why, in time, I have no doubt he will be baptised and confirmed into whatever church this is. As may I, eventually, perhaps, so that I can take Communion, which is the only time I feel a bit left out. Kitty will probably not. I don’t know about my wife. I don’t even know how she votes. Or what she sees in Queer Eye.

I’ll still be a Jew, in the way that a black man would still be black. I’ll still use Yiddish words where English ones won’t do and say “his mother was Jewish, you know” whenever Harrison Ford comes on screen. But I’ll be a Christian one.

I will never forget my father’s tears as he said kaddish for his dad (a mitzvah I was unable to perform for mine when the time came), convulsed with grief next to a muddy hole in a bleak Jewish cemetery in Hertfordshire. But I do not see my churchgoing as a betrayal of that. Far from it. I see it as a continuation. A picking up of something that was dropped. Because in the end, we’re all just ash, aren’t we?