"I had never read a book in my life. I’d barely even touched a page, except to confiscate my brother’s novels so he would hang out with me. That impenetrable realm where awkward poindexters escaped into ludicrous worlds of printed fantasy, books and I just never got along.

Naturally I was surprised when I was asked to judge a book prize. The judges were assembling a panel of illiterates and I agreed to join them: not because I actually cared about whether books contributed to culture, but because I wanted to reconnect with my family of unashamed bookworms.

The first book I received was Ulysses by James Joyce, an eccentric Irish alcoholic who neglected his wife for six years while writing the novel. After suffering a few pages of this unfathomable word soup, I moved onto to Golding’s Lord of the Flies, an action-adventure about schoolboys. I noted, “Surprisingly enjoyable. Nice adjectives. Like the allegorical dialogue,” but then ran out of things to say. Maybe that’s part of the reason there is so little cultural discussion of books: there simply isn’t much to talk about. Lord of the Flies is clever and snappy, but it’s not exactly Bioshock. It’s not even Call of Duty."

It is ridiculous to criticise video games by comparison to literature, film, music, or interpretative dance. Unfortunately, Lucy Kellaway recently chose to evaluate them through this narrow cultural lens. As a judge for the GameCity prize, she was tasked with playing some of the year’s most critically acclaimed titles including Journey, Fez and Proteus. To be fair, she gives them all a try, concluding: “This experiment has made me realise how bad I am at being bad at things”.

Dara O’Briain once quipped that video games were the only medium that actively withheld its contents from the beholder: you would never have a progress-impeding quiz at the end of a book chapter, or be denied access to the next track of an album until you fully understood the previous music. While I would never mock Kellaway for being bad at games, I question her motivation for taking part in the GameCity judging process. She openly admits she doesn’t care about games’ contribution to culture. Like any other art form, it’s just not fair to approach games with a bad attitude and a mind full of clichés.

Deriding a video game for its primitive narrative misses the point. You wouldn’t read a book and complain about its lack of interactivity or longevity. While some games do have an engrossing story, it’s not an essential prerequisite for greatness. I once wrote a satirical book review in the style of a game article to highlight the facile nature of video game criticism, but there’s another point to be made: we can’t criticise one medium for not living up to the expectations of another. This art gallery is boring; the pictures aren’t animated. This opera is awful; I can’t understand the plot. My girlfriend played Journey (voluntarily, I hasten to add) for the first time last weekend: she’s not a "gamer", and she enjoyed it so much she wants to play it again. The difference here is in the intention, not the emotional reaction.

But what’s most infuriating about Kellaway’s piece is its underlying truth: most video games really don’t have a lot to say. They generally have the lyrical nuance of a Eurodance song, and even a game like Spec Ops: The Line can’t properly critique the horrors of war when the player herself is actively creating those horrors. Kellaway’s favourite game from the selection was Journey, one of my favourites too. But I also love Hotline Miami, an ultra-violent and sadistically challenging title where you dress up in an animal mask and murder gangsters in a hallucinogenic world. I’m not even ashamed - it’s a brilliant game! - but I wouldn’t pretend it had reached a zenith of cultural significance. Video games were born as entertainment, and like cinema they have matured beyond this role, but the existence of the Cannes festival does not preclude the continued success of Hollywood. Games exist on a broad spectrum from the outrageous Borderlands 2 to the deeply personal dys4ia.

It’s sad that the most popular and heavily marketed titles are superficial junk food for the brain, like Call of Duty and FIFA. I loved The Raid, but I wouldn’t watch it with my granny, and likewise her only knowledge of games will be an advertisement awkwardly sandwiched into a break during Coronation Street. She has just broken her hip and a Nintendo DS would stave off the boredom of rehabilitation, but you can’t give a games console to someone who struggles with a television remote. I emphasise with Kellaway’s feelings of having inadequate dexterity: video games are often unashamedly elitist and obtuse, which makes the work of the charity Special Effect for users with disabilities even more praiseworthy.

Like many games, gaming culture itself is often tasteless and offensive. "Booth babes", casual sexism, homophobia, transphobia: you name it and it will be there, unveiled and unpleasant. In many ways 2012 has been a watershed year for gaming discourse where the community has openly challenged its worst elements, but there’s still a mountain of intolerance and cruelty to climb before gaming culture is a place where all feel welcome. Sometimes I feel like I’m defending the indefensible. Yet pointing at Call of Duty and saying “look at how worthless these games are” is like playing the new One Direction single as evidence for the death of music as art. Although maybe that’s true.

As an self-professed adventurer of digital fictions, it hurts to admit that much is rotten about the state of video games. But the way to improve this cannot be to dismiss games as “not all evil”, which suggests that most games are evil as if Satan himself was distributing the software. Is that why you can’t play a game backwards? Even if Kellaway doesn’t care about their place in the cultural conversation, I do, because they are as much a reflection of culture as anything else. GameCity is important: quite frankly, games deserve a better discourse than what is currently on offer. We need more criticism that is intelligent, personally reflective and nuanced; treating games more like experiences and less like gadgets. We need to critically examine the role of games in culture, because we can only demand more from our media when we understand where they fall short. Sure, video games are fun, but they’re serious fun.

Kellaway’s final insult is the classic derision of games as a waste of time: she has no time to play them when “there is life to be lived and books to be read and emails to be written”. But even if you lived in a cave with only books for cultural sustenance, you could never read them all. There will always be a choice of indulgences, and I find that quite thrilling. As a good friend puts it: “time you spend doing something you enjoy is never wasted”. And to be honest, if I miss out on reading a few books because I’m playing Journey, that’s a life well wasted.

Alan Williamson is editor of Split Screen. He tweets as @agbear.