It feels strange to enter a gallery of contemporary art beneath a massive classical portico. Yet there is a kind of inevitability in Charles Saatchi's journey from abandoned factory to classical pudding.

Saatchi's first gallery was poetically situated in a former paint works, and in 1985 it was among the first of London's industrial art spaces. It was designed by American architect Max Gordon, superbly connected with the New York art scene. London's art remained confined to the prissy galleries of Cork Street or the second-rate classical public galleries: the bright light and white walls of Boundary Road blinded with their cleanness, blew up the scale and allowed the mostly American, mostly big art to glow.

By the time Tate Modern opened in 2000, the hulking industrial carcass had become a universal language. Saatchi, contrary as ever, shifted over in 2003 to County Hall, an overblown Edwardian piece of municipal grandeur that was an odd choice for a gallery. If former industrial spaces held a sentimentality for the means of production being replaced by art, County Hall revelled in a lost civic ambition, an era when cities expressed themselves through civic architecture and not retail. It was another dead building type. But they were always poor spaces for art.

Now, in a structure built as a school and subsequently used as an army training facility, Saatchi has found a further level of defunctness. One side of the military-industrial complex has already been appropriated by art, it seems appropriate that the other side should start going the same way.

Dating from 1801 the building is a self-consciously grand pile presenting a daunting neo-classical portico to a grassed sports field and the track on which Roger Bannister trained to break the four-minute mile in 1954. Inside it has been municipalised, functional rather than grand, and its blank, useful spaces present a fine framework for the insertion of 15 new galleries. The gallery is the work of architects AHMM, who spent three years searching with Saatchi and property consultants Pilcher Hershman.

In spite of its classy, classical setting, there is nothing twee here. The architects have given Saatchi back some of the stripped brilliant whiteness of Boundary Road. The galleries are clean, sparse and huge. The ceilings have been sucked back into the structure, leaving beams and roof structures exposed, squeezing every inch of space out of the volumes. They are top lit and self-contained, in here the art is everything.

The route through the galleries takes you straight from one space to the other. Unlike a traditional public gallery there is almost no circulation space and the effect is intense, a serious, unrelieved dose of Saatchi-promoted art. The big conceptual works and installations look good, paintings work as well as sculpture and the range of spaces, from sub-industrial with exposed services and roof trusses to minimal modernist with clean flat ceilings and heavy illumination allows the hugely varying works to shine.

The architecture here is all in the organisation, the volume and the route; there is virtually nothing else. AHMM has done a fine job suppressing any urges it may have had to express anything. It is, in a way, a perfect gallery, a formless parasite inhabiting the husk of a historical hulk. Saatchi's smooth surfaces present the most neutral possible background, where even two centuries of history are not allowed to intrude. Jackie Wullschlager's review of the new work in the Saatchi galleries will appear in FT Weekend on Saturday