It is a myth central to the survival of the publishing industry that essential fiction is produced in fairly large quantities each and every year. The annual lists of must-read novels, stories, or anthologies only ever get longer, as if the more we print (naturally!) the more essential fiction will be written, discovered, and nurtured. It is a fallacy, of course—great writing is not produced on a quota basis, cannot form a reliable percentage of the total words printed in one year—but it is one that is eagerly embraced by a usually sceptical audience. Readers validate their hobby of choice by pretending that there's always a new classic to hitch onto. The sad truth is that there isn't. Great fiction is as rare now as it ever was—which is considerably more so than we sometimes like to believe.

And so the traditional "Year's Best" anthologies which are churned out every twelve months in the SF field, and purport to present to us the most essential stories of the last year so as to keep us up-to-date with all the very latest innovations and creative explosions, should be approached with an eyebrow circumspectly raised. All these books have to choose from, after all, is twelve months' worth of stories. Even in a short story field as plentiful as science fiction's, such a constrained pool is unlikely to find Gardner Dozois his 300,000 words of superb fiction for St. Martin's Press. It might, for instance, find him three or four stories everyone should read, and leave him with a quarter century that are filler or worse. Hartwell and Cramer, over at EOS, will find themselves in a similar bind, although they will at least have the decency not to be quite so verbose in proving the point; Strahan and Horton (the latter of whom is new to the "year's best" shindig), anthologising for Locus Press and Prime Books respectively, and also keeping their page count down in the process, might have a little more success in keeping the dross-to-fairly-OK ratio on the right side of "meh," but will be equally beset by the inherent problem of their chosen endeavour. We might have read an awful lot last year, but let's not pretend that we were reading timeless classics as we did so. For the most part, the "year's best" is the "decade's largely forgettable."

Gardner Dozois is of course the granddaddy of the annual anthology—this is his twenty-third such volume. His housebrick of a collection gathers together 29 stories from a fairly wide variety of outlets (albeit with a slight tilt towards the traditional magazines) and is the closest thing the field has to a single "canon-forming" volume. The warning bells should ring, then, when in his introduction Dozois time and again characterises 2005 as a "decent if not exceptional" year: its novels weren't as good as 2004's, its anthologies were average, its genre movies made a lot of money but not a lot of sense. Most of the short stories he reprints have a similar air of almost-but-not-quite about them.

For the most part, Dozois errs on the side of conservative narratives with traditional preoccupations: Alastair Reynold's space opera puzzle "Beyond The Aquila Rift"; Robert Reeds's space station gumshoe drama "Camouflage"; the ever-present Michael Swanwick's light time-loop tale "Triceratops Summer"; Dominic Green's cautionary "The Clockwork Atom Bomb." Predictably, all the big names are here—Ken MacLeod, Bruce Sterling, David Gerrold, Stephen Baxter, Gene Wolfe, Joe Haldeman. New writers get shorter shrift (though one of them, Daryl Gregory, provides one of the collection's most interesting stories), and some of Dozois's other choices seem merely odd—is Liz Williams's "La Malcontenta," for instance, really one of the year's best stories, or is it merely one with which the editor is comfortable? This vignette—to call it a story would be overdoing it—is slight, over-grand and reads rather like something from the 1970s—all stern matriarchies and banished males. The reader is left curiously without fresh illumination at its close, but in its almost retrograde quality and po-faced prose it is terribly fitting of Dozois's often old-fashioned, sometimes out-of-touch, frequently over-worthy collection.

Dozois gives us what we expect of him—a big, hefty collection of the year's most recognisably science fictional writings. It is a formula which makes for often dull reading. It is not that he doesn't share some of his stories with others of this year's editors—Strahan, like Dozois, reprints McDonald's superlative novella "The Little Goddess," Paolo Bacigalupi's impressively imagined "The Calorie Man," and Sterling's probably-mad-possibly-inspired "The Blemmye's Stratagem," for instance, whilst Hartwell and Cramer duplicate Gregory's excellent "Second Person, Present Tense," and Rich Horton has Swanwick's effort plus Mary Rosenblum's "Search Engine." It's just that, in its length and almost unremitting conservatism, Dozois offers a "year's best" which is deadeningly canonical, almost bravely uncourageous.

Rich Horton is a different kind of editor, as interested in the edges of the genre as he is its core, and muses in the introduction to his own collection that, "I don't think new ideas are as common as they once were, but fresh treatments of old ideas, even explicit homages to old stories, can be just as exciting." This recognition that, when the novelty of invention is absent from an SF story, then all it has left is that dreaded tool of mimesis—the innovative treatment—perhaps explains why Horton's choices sometimes seem fresher than Dozois's even as they tread the same water. Horton reprints Susan Palwick's spritely, amusing "The Fate of Mice"—a familiar tale lightly told which still has more joy and fun in it than whole swathes of Dozois's choices. Again, it is not an eternal classic, but it is nevertheless a story with that freshness of which Horton writes, and it is that which saves it. Similarly, Howard Waldrop's "The King of Where-I-Go," an ostensibly old-fashioned time-hopping story, has a modern prose sensibility which makes it feel contemporary and wise, and Douglas Lain's wonderfully wry "A Coffee Cup / Alien Invasion Story" consciously presents a New Yorker SF story for our times.

But enough such stories are not published in twelve months to fill a collection, and Horton also prints "Bank Run" by Tom Purdon (a nice idea catastrophically poorly written) and "The Edge of Nowhere" by James Patrick Kelly, which tries its best to be interesting but forgets to be engaging. The latter is also a story published in EOS Books' "year's best," edited by David G. Hartwell and Kathryn Cramer. It is a story more typical of their anthology than Horton's: a story which lifts off from a dense, even difficult, idea and then seeks to construct a narrative which explains it. There are post-humans here, and talking dogs, and self-consciously amusing characters described in deliberately cute turns of phrase; the story has the word "cognisphere" and utopias and libraries. And yet its prose is singularly unable to cut a readable path through the high concept: "Lorraine Carraway scowled at the dogs through the plate of the glass window of the Casa de la Laughing Cookie and Very Memorial Library," is its first sentence. Hartwell and Cramer duly reprint it.

Given their collection's interest in concept over content, then, it is unsurprising that it falls back time and again on the series of decidedly short stories published in the scientific journal Nature last year (no less than 10 of the collection's 31 stories are taken from this outlet). These short shorts, perhaps because of their length or perhaps because they are meant to be abstracts, never quite fulfill their potential—even the peerless Ted Chiang fails to really hit the target in his "What's Expected of Us." All the Nature stories provide food for thought, but the sustenance they provide the imagination is more meagre. By and large, this is true of Hartwell and Cramer's collection as a whole.

There are some high spots for Hartwell and Cramer—Daryl Gregory's success has been noted, and his story lights up this collection with wit and intelligence. Examining the mysterious distance between thought and consciousness, "Second Person, Present Tense" introduces us to a drug-addled young woman who has lost her old personality and had it replaced with a new one which is no less "her" (for, in this story, our essence is to be found not in our identity but our neurology). Smoothly but discomfortingly written, the story's concept—that consciousness isn't the controller but the controlled—is intelligently explored and then, crucially, spun out in thoroughly human directions. In addition, Rudy Rucker's mischievous "Guadalupe and Hieronymus Bosch" is zesty and daring, whilst Bud Sparhawk's tight and nicely conveyed "Bright Red Star" offers more traditional joys, and Hannu Rajaniemi's "Deus Ex Homine" is headily imagined stuff. But perhaps more than the other "year's best" anthologies, the Hartwell/Cramer offering falls foul of itself rather than the limits of its remit. Stories like "Rats of the System" by Paul McAuley, "Mason's Rats" by Neal Asher, and "A Case of Consilience" by Ken MacLeod are simply moribund things, too much in love with their own conceits and with too little interest in what they might mean.

To compare MacLeod's story to Palwick's "The Fate of Mice," reprinted in both Horton and Strahan, is instructive: both are explicit homages to previous tales (MacLeod is riffing on Blish's 1953 first contact classic "A Case of Conscience," Palwick, of course, on 1959's "Flowers for Algernon" by Daniel Keyes); both are relatively short; both are rather consciously "modern" in their take on such hoary old science fictional themes. But where Palwick is funny, playful, and arch, MacLeod is leaden, careful, and portentous. MacLeod's Big Idea has grown so fat that it has flattened his story (or perhaps it never had anything as meanly corrupting as an actual story—it's hard to tell).

Most of the stories published in these anthologies aren't actively bad (though some are approaching it—Dozois's own "When The Great Days Came", or Cory Doctorow's "I, Robot," both in Hartwell and Cramer's book—although Strahan reprints the Doctorow, too—lack much in the way of redeeming features). It is simply that they're not great—they're OK, they're passable, they're getting there. And yet, the collections still sell. Perhaps the people buying them are devoted to the canon; perhaps they really do believe that these volumes are overflowing with fine writing, new ideas and exciting narratives. They must not be mistaken for that. Indeed, because of their size, number, and editorial idiosyncrasies, these "year's best" collections can at times be depressing reading. This is the best SF did in 2005? Shouldn't it be slightly less wordily pleased about that?

The best of the four anthologies discussed here is Jonathan Strahan's, for Locus Press. The similar annual series Strahan co-edited with Karen Haber for ibooks ended last year, and the current volume is a transitional one before Strahan moves to a new publisher next year. We are lucky to have it—the collection proves to mix the traditional and the innovative in a generally satisfying manner. If Horton's choices are sometimes a little flighty, and Dozois's a little safe, Strahan ploughs the middle course. Thus, James Morrow's impish "The Second Coming of Charles Darwin"—in which an evangelical Christian group sponsor a time-travelling scientist to return to the Galapagos and eliminate the specialised genes of Darwin's subjects—sits side by side with the more straight-ahead space opera of Gwyneth Jones's dirty and dense "The Fulcrum" and Robert Reed's re-examination of post-human immortality, "Finished." There are slips, of course—Strahan fills some space with Pat Cadigan's uninspiring vampire chuckle, "Is There Life After Rehab?," and he also reprints the lacklustre Swanwick and Doctorow stories, but these actually add to my sense that Strahan is providing a true cross-section of the year's SF: some experimental, some literary, some pulpy, some epic, some small, some good, some bad.

Strahan also manages to print both of the year's truly best short stories (and, to be fair to him, so does Dozois): Ian McDonald's evocative "The Little Goddess" and Vonda M. McIntyre's daringly ambitious far future tale, "Little Faces." McDonald's story centres on a young girl who first becomes divine and then is cast down again into the ranks of the mortal and profane. A companion piece to his superlative 2004 novel River of Gods, set in the same fractured future India and written with the same verve and sense of place, "The Little Goddess" is the best story of the year and—yes, OK—a candidate for classic status, even if it may feel to some as if the story's true impact is diminished by the more thorough earlier treatment. McIntyre's story, set in an incalculably distant and alien future, features human beings who share their love for one another, and reproduce, through parasitic creatures grafted onto the bodies of their lovers. With very often only these companions for company, these post-humans live for millennia—frequently in suspended animation for centuries—in the exclusive confines of their personal vessels, making their own journeys across the cosmos and meeting each other only sparingly and irregularly. The story examines loss and sexuality, and love and grief, tenderly and unblinkingly, piling idea upon idea in so layered and assured a way that the narrative always remains paramount. It's a genuinely impressive story.

Strahan also has Reynolds's twisted-but-occasionally-static Stapledonian mini-epic, "Understanding Space and Time" (another referential tale which adds twists to its antecedents rather than grovels to them), Bacigalupi's compelling but thinly characterised "The Calorie Man," and Sterling's bewildering, confounding, and quite possibly barking mad medieval romp featuring assassins, nuns and Blemmyes—all of which are fine stories with only a half flaw or two. There are few essential stories he misses—David Gerrold's "In The Quake Zone" offers a few promisingly idiosyncratic observations but thunderously formulaic everything else, for instance, and thus can be found in Dozois, whilst the Locus collection also misses Daryl Gregory's gem (this represents Strahan's biggest miss, since he can be forgiven for passing on the irreverent-if-wonderful Douglas Lain invasion tale which Horton reprints). This gives us six stories: McDonald's, McIntyre's and Gregory's, plus a second string of Bacigalupi's, Reynolds's, and Sterling's. Strahan's selection (or rather half of it) really is the closest to a true "year's best" we have.

In his helpful little introduction to Prime's anthology, Horton observes that many of 2005's stories could be categorised in one way or another as "hard" science fiction. Whilst definitions of what is and is not "hard" SF may vary, what Horton says is close to the truth of it: though many of the year's offerings were in many ways retreads of old ideas, such was the profusion of ideas and ideas-based tales that it was difficult to find a single unifying theme which preoccupied a large number of writers in 2005. Genetic engineering crops up often enough, but so do contextless space operas. Times of war, of course, recur in some of the stories, but so too do cheeky little genre in-jokes. On the evidence of these collections, 2005 was a diffuse year, and the ways in which the editors chose their stories based more on form than content would seem to support that supposition. Strahan's phrase of summary for the year is "interesting and impressive"—the scope of the year's imagination is indeed both, with a wide variety of ideas, tropes and themes at play.

What is happening in these four anthologies, then, is a distillation: the best treatments of science fictional ideas are bubbling to the canonical surface; the minutes of the dialogue are being taken. But this does not necessarily make for great stories, or great anthologies. We see Reynolds and Swanwick with four mentions each (the latter with two stories, the former with, amazingly, three); we see Asimov's and Nature emerge dominant in terms of the coveted "most stories reprinted" tag (though Nature's success rests almost entirely on Hartwell and Cramer), with Analog, a by all accounts newly invigorated Interzone, and the non-print Strange Horizons also worth honourable mentions. We see a themeless twelve months, happily bereft of a single topic harped on about by more than two or three authors. We see the Internet—Strange Horizons, Amazon Shorts, the sadly defunct SCIFICTION—make its presence known, and we see too slight a glimpse of what newer writers are saying. We see, for better or worse, the genre in action.

But the "year's best" of SF—that is, the actually decent fiction—is right there in Strahan, consisting of around half his volume's page total. The genre and its scions perhaps need all four volumes as a record of the year's suppositions; the general reader categorically does not. Six great stories, after all, is good work for twelve months. It's just really hard work over 2000 pages.

Dan Hartland is a freelance creative thinker figuring out what to think. A writer and musician of the inverted commas variety, he splits his time equally between these two things and procrastination. He comes to science fiction from outside the genre, and is a little too happy to remain a gadfly.