Doctor Christopher Andrew, a fellow and senior tutor in history at Corpus Christi college, Cambridge, has written a book – a big book – called Secret Service: the Making of the British Intelligence Community.

It is the first scholarly study of the subject. Previously the feeling among historians was that the topic was far too shrouded in mystery (and thus myth) to be studied. And that the popular image of the James Bond-type secret agent and/or the startling revelation school of journalism augured badly for a serious and non-sensational approach.

In his room at Corpus (with the sun shining over the quad and the distant cries of "Howzaat" being carried on a light breeze from Fenners, etc.) Doctor Andrew told me:

'The great problem for anybody interested in 20th century history is the enormous over-production of paper that's the curse of the 20th century historian. The British Intelligence community has destroyed or hidden most of its paper, its archive. Even though things which are officially secret are, curiously, available. What managed to escape the official censor is quite enough for a scholarly study. Paradoxically the attempts of the authorities to make the history of the Secret Service unwriteable have made it more writeable than if they'd let everything out.

"I'd shared the view of most historians that there wasn't material for a book but one could write about particular episodes. The first thing I wrote was about the Zinoviev letter – the intercepted Comintern (Communist International) message in 1924 which discredited the Labour Government and possibly assisted in them losing the second election of that year.

"In looking at what other people had written on the subject I realised that they'd all made what seemed to me to be an error. They failed to realise that intercepting communications is something that the Government does everyday and if you're going to understand the Zinoviev letter or any other communication, you've got to interpret it within a much larger day to day pattern of message interceptions, decoding and so on.

"Having done that I realised I could write more. The discovery that I could write the complete history was a progressive revelation.

"Until now the subject has been left to non-professional historians (cue Chapman Pincher) and the problem has been that there is no professional historian to say to some preposterous allegation where's the evidence? Where's the footnotes?

"Secondly professional historians had been put off by the James Bond idea that intelligence consists of one dazzling coup which changes the course of history.

"People think of cloaks and daggers and dramatic coups but all that intelligence is, is information. And secret intelligence is something which policy makers reckon they need which they can't get from conventional sources and it's going to require a spy, a codebreaker, a spy satellite or whatever other covert source to get hold of it.

"Once they've got it, it can be even more difficult to use it."

Indeed, the book reveals a startling series of blunders in the use of intelligence. The modern Secret Service has its basis in the notion that Britain was teeming with German spies in the run up to World War One. An idea that was later shown to be quite erroneous.

Conversely, it was an outstandingly well used piece of intelligence – the breaking of the German Enigma code during World War Two in the operation known as Ultra which is reckoned to have shaved several years off hostilities – which has now to some extent altered public conception of the Intelligence Service.

The existence of Ultra remained secret up until 1975 when it was de-classified. An act which Doctor Andrew considers to have been "the lifting of the corner of the veil".

"The reason Whitehall argued strongly against letting that secret out even 30 years after the end of the war was that it would be the thin end of the wedge. Once people were told what codebreakers had achieved in WW2 then they might ask what they are achieving now.

"The interest caused by Ultra was one of the things that increased public curiosity about GCHQ so I think people are just beginning to realise that there are completely other areas to intelligence gathering than what they might have thought of. But probably the area that's attracting least attention is how they actually use the stuff."

The most recent revelations have involved the activities of MI5. As we speak, the BBC vetting 'scandal' has just broken. One of the ironical twists being the admission by the BBC that they started vetting in 1937. Just at the time Guy Burgess began making programmes for them.

"Burgess played a series of practical jokes on the BBC which I believe to be without precedent. In 1942 he commissioned a talk from a member of Soviet Intelligence Service. Included in the broadcast was the sentence '...and the Soviet Intelligence Service is one of the best in the world'.

"I think Burgess must have just about fallen off his stool when he heard that. Blunt and Philby must have felt that if Soviet Intelligence could get away with that – broadcasting on the BBC about how good they were – there was nothing they couldn't get away with.

Burgess, Kim Philby and Donald Maclean formed a triumvirate of 'moles' (ie Soviet plants in British Intelligence) the discovery of which (mounting evidence confirmed when they bolted for the Soviet Union) in the early 60s caused a bit of a stir, to say the least (more recently revealed was Anthony Blunt, the so-called fourth man').

Their 30 odd years of penetration had reached a point where Philby was being hotly tipped as the future 'C' (head of MI6) and the high-flyer Maclean bound for the chief's chair in the Foreign Office. At the time of their departure both already held key posts. Philby as MI6's liaison man with the CIA and Maclean as head of the F.O.'s American desk.

"I'm sure the Soviet Union is still trying to recruit moles. Until the Bettany affair I would have said it was quite impossible for them to recruit nowadays as they did in the 30s. What is striking about the moles recruited from this University (the above named) is their extremely high intelligence and idealism of their initial moves.

"Most of them didn't actually realise they'd joined Soviet Intelligence. They thought they were doing intelligence work for Comintern. They only discovered bit by bit that what they'd joined was a front for the KGB. But they thought the Soviet Union was the future, that British society – Cambridge between the wars – had passed, and that the way to defeat fascism was by engaging in a secret struggle against international fascism under communist leadership.

"That kind of recruitment is clearly impossible nowadays. It stretches belief that anyone could go through a good liberal education system and end up believing that the Soviet Union is the hope of mankind. It's highly significant that the most recent moles that have come to light have been far less able people, not recruited through idealism but in most cases by a result of personality defects, sex, money or whatever.

"Bettany worries me a little bit but it's very difficult to read what was published about Bettany since he was sentenced and believe the man had a normal personality. So I'm not inclined to believe Bettany is a serious exception to the rule about the difference between moles in the 30s and moles in much of the 80s.

"But there are a number of things about Bettany which I still find very curious..."

A chief conclusion of the book is that the lack of parliamentary accountability has led to inefficient management of the Secret Service. In 1977 Doctor Andrew put the case for a parliamentary select committee on Intelligence. The Times noted 'it was as if he had made a disrespectful remark about the Royal Family'.

"Despite all the improvements of the last three quarters of a century, we're still in a position where the Government hasn't begun to define a credible line for the bounds of official secrecy – which means that even the name of the theatrical supplier where the head of MI6 bought a disguise in 1909 is still classified and there are many other examples of idiocies like that one.

"The Government is not accountable to parliament in any way for its management of the Intelligence community. It's utterly preposterous but that's the way it is. The greatest weakness of the Intelligence community which are frequently blamed on the Intelligence community chiefs, are often derived from the inefficiency of Government management. It's inefficient because it's not answerable.

"If you look at the story between the wars, it is one Government cock up after another. Through sheer incompetence they gave away the best intelligence Britain possessed, the intercepted decoded signals of the Soviet Government. British Intelligence was run down to a point that threatened national security and security at Whitehall was so farcical that it was penetrated by a series of Soviet moles. The Italian Secret Service was able to get documents with ease from the British Embassy in Rome and security at the British Embassy in Berlin was a laughing stock. The lack of accountability between the wars was a recipe for recurrent incompetence."

How will the book be regarded by the Intelligence community themselves?

"Unfavourably, I would expect. One very senior and recently retired civil servant wrote me a letter after he had looked at some of my book and said 'the intelligence bureaucracy should be very grateful to you. I can assure you that they will not be'.

"I think that those who run the Intelligence community will feel the book is a very bad thing. Their attitude to public debate of the Intelligence Service is roughly the same as the attitude of a Victorian spinster to a discussion of sex at a Victorian dinner party. It's an outrageous breach of good taste to even talk about it at all.

"There are perfectly rational arguments for not talking about current operations but it's the kind of taboo that those who've grown up with it will never shake off. There are many other members of the Intelligence community who have come to the conclusion that there needs to be some sensible definition of the limits of official secrecy and some sensible method of accountability.

"But anyone who takes the view of the present Government is bound to regard my book as deeply unpatriotic, deeply subversive and a thoroughly bad thing. That shows how silly they are."

While acknowledging that MI5 probably have a hefty file on him ("I don't mind as long as they get it right") and that they might well be monitoring his phone calls ("I don't mind as long as they don't tell anyone else'), the level of actual hindrance encountered in researching the book has been minor. Doctor Andrew still enjoys walking down alleys on dark, foggy nights.

"I think our side is more decent than the other side. In the end their attitude to the level of secrecy which they need – banning World War One documents – is wholly dotty. They're dotty rather than malevolent in their attitude to people like me. They dislike people ferreting around with all this stuff but the idea that my book when placed in the middle of Red Square will do anything other than confuse the KGB is pretty improbable.

"There have been attempts, which I think are mildly disdainful, to persuade people in their eighties not to talk to me about things going back as far as the 20s. A number of them, I'm delighted to say, ignored the official advice that they shouldn't talk to me. MI6 or those who issued the warnings made fools of themselves on one or two occasions... but they'll grow out of it."