The new display at Tate Modern of Gerhard Richter's large, abstract "Cage" canvases from 2006 is as shimmering an exhibition of paintings as any London has seen this century. Go at once: Tate Modern is "refreshing" its level three and five galleries, as it does every spring, in advance of the UBS (NYSE: UBS - News) Long Weekend at the end of May, and the Richter room now is relatively quiet. Richter is a painter of almost velvet stillness, flamboyant only by stealth, whose self-consciously late, solemnly glorious work demands peace and long viewing.

The six paintings are composed in his characteristic swiping, blurred style of over-painted and obliterated layers, fine-tuned nuances of grey and white worked through with coruscating colours - carmine, emerald, turquoise, cadmium yellow, fiery orange - dragged across the canvas, smeared, wiped, leaving fragments of beauty on cool but sensuous surfaces. They suggest rain and mist, instability and displacement, absence and endings, classical rigour and postmodern ruin. They echo the northern European palette of earnest darkness and piercing brightness that goes back to Grünewald and Caspar David Friedrich, but Richter is also a minimalist, a denier of meaning, ideals, personal signatures. He has named the works in honour of composer John Cage, in reference to his Lecture on Nothing - "I have nothing to say and I'm saying it."

Like all Richter's work, "Cage", if it is about anything, is about painting's future, a theme of particular resonance at Tate Modern. At the moment, the museum's permanent collection is far outstripped by those of its global rivals, New York's Museum of Modern Art and Paris' Pompidou Centre. That is because MoMA and the Pompidou bought wisely in the 20th century, before Tate Modern existed. Now that is history, and so is Britain's former cultural deference to America and France. Today's art is global. By the time Herzog & de Meuron's spectacular extension to Tate Modern opens in 2012, the museum will be judged rather on its 21st-century collection, on what it begs, borrows and buys in the next few years.

That is why the annual spring refresh, trumpeting recent acquisitions and curat-orial tastes, is so indicative and important. The Richter makes a statement: on the artist's website, a bombastic video opens with a panorama of Tate Modern and the Thames, underlining the museum's appeal to artists. The video plays to the accompaniment of Cage's minimalist "Four Seasons", which puts Richter's delicate, light-suffused paintings of fleeting moments in the context of his American contemporary Cy Twombly. Twombly's great "Four Seasons" also hang at Tate Modern, with Joseph Beuys' installation "Lightning with Stag in its Glare", which in turn sends you to Richter's important early grey blur painting "Stag", and back to ideas of European history painting. "I can't paint as well as Vermeer: we have lost this beautiful culture, all the utopias are shattered, " Richter said recently. "I still want to paint something like Vermeer but it is the wrong time and I cannot do it."

To make possible such myriad associations, endlessly questioning art's stories, is how world-class museums work. The Richter room is not only stunning, it gives new resonance to familiar highlights in the collection, which is why acquiring top contemporary pieces is essential to the entire Tate Modern project. In recognition of this, Louise Bourgeois' giant spider "Maman", Damien Hirst's bisected cow and calf "Mother and Child Divided" and David Hockney's 40ft landscape "Bigger Trees Near Warter" have been given to Tate by the artists in the past six months. Yet Tate's tiny acquisitions budget and woeful purchasing decisions mean that the gap between the charismatic pieces it is gifted or loaned and the dire mediocrities it is currently buying is becoming an embarrassing chasm.

The new displays highlight this all too well. Among the temporary loans are a taut, lovely abstraction by Joan Mitchell, a great artist so far unrepresented at Tate, and a group of compelling portraits, restrained yet somehow fantastical in their retro elegance, by the Malian photographer Seydou Keita, lent by Geneva's Contemporary African Art Collection. The new purchases, on the other hand, are dowdy, intellectually dull and art-historically insignificant.

The wooden contraptions and kebab of soap by Miroslav Balka and the foam mask by Pepe Espaliaú cross Ikea with a post-Auschwitz aesthetic to dreary effect in the "Poetry and Dream" section. The gallery containing "Projection Room 1971-2006", Paul McCarthy's video of 35 years of puking, squelching and splattering spews bored visitors out as soon as they enter; Marlene Dumas' fashionable, politically correct 1990s portraits turning on race and gender identity ("Lead White", "Ivory Black") already look dated. Hélio Oiticica's origami-like coloured sculptures are child's play alongside the ethereal, floating, serious Ellsworth Kelly abstractions in the next room. The overrated Susan Hillier's "Psi Girls" is irritating, post-Victorian whimsy on film. Only "Thirty Pieces of Silver", Cornelia Parker's luminous, poignant steamrollered plates, teapots and trombones suspended in hovering disc-like formations, in a gallery already attracting long queues, shines out as an inspired purchase - but that was bought in 1998, and first shown at Tate Britain.

Tate cannot, of course, alter rocketing art prices, but it can exercise quality control at all levels at which it buys. Its job, director Nicholas Serota said recently, is "not just to show household names but to create household names". Balka, Oiticica, Hillier, are not among those that need to be created.

Tate's international standing and popularity are its trump cards in attracting loans and gifts of Richter-like status. If it continues to round up cheap Polish and Latin American installations and tired videos, however, Tate Modern risks losing its chance to become the world's leading 21st-century museum and looking instead like the world's saddest car boot sale.

www.tate.org.uk