The ultimate insider who knew the royals’ darkest secrets: Accused of masterminding a plot to murder Diana, Robert Fellowes was at the centre of the family’s most private controversies, writes RICHARD KAY

You could spot him most days if you knew where to look. An angular figure pedalling his bicycle from one palace, where he lived, to another, where he worked.

Unassuming and anonymous in his cycle clips he may have been, but for 22 years Robert Fellowes knew more secrets about the Royal Family than perhaps anyone else in Britain.

As the late Queen Elizabeth’s private secretary, he was at the epicentre of every crisis that engulfed her during the most perilous period of her reign, from the marital and domestic upheavals of her children, to the disasters and tragedies of the Windsor Castle fire and the death of Princess Diana.

Throughout it all, he remained a model of correctness and duty who appeared to have stepped from a gentler age. It was often said the Almighty must have been having fun making him related to the Queen’s two troubled daughters-in-law — Diana, who was his wife Jane’s sister, and Fergie, a cousin.

His enemies saddled him with a grouse-moor image and sneered at what they called his ‘tweedy amateurism’.

Robert Fellowes, private secretary to Queen Elizabeth, with her late majesty

Robert with Prince Charles in 1993

But behind his owlish spectacles and frayed cuffs, Lord Fellowes, as he became on retirement from royal service, was a shrewd and effective aide who often displayed remarkable sang-froid.

That coolness under fire was put to a severe test when accused by Mohamed Al Fayed of masterminding the plot to ‘murder’ Diana and his son Dodi.

The former Harrods owner was convinced that the car crash in Paris in which the couple were killed was an MI6 assassination — and that Fellowes was involved.

He claimed that the Old Etonian courtier had helped co-ordinate the ‘conspiracy’ by commandeering a section of the British Embassy in Paris, from where he sent messages to GCHQ shortly before the fatal high-speed crash.

When he gave evidence at the High Court inquest into the princess’s death, Fellowes said that far from being in the French capital on the night of the tragedy — August 31, 1997 — he and his wife had visitors at their Norfolk home and had spent the evening in a Burnham Market church hall at a talk by Rumpole of the Bailey creator Sir John Mortimer.

Significantly, Fayed’s lawyer did not challenge this account.

For all his old-world courtesies, Fellowes was also surprisingly tough. Shortly after Diana’s death, the Queen and Prince Philip had been swept up in some undiplomatic exchanges on an official visit to Pakistan and India, thanks to the incompetence of the then-Labour government’s foreign secretary Robin Cook.

A s the row escalated, India, which resented Cook’s offer to mediate over the disputed territory of Kashmir, cancelled a speech the Queen was due to give at a banquet in Madras. In a pointed and rare public intervention, Fellowes declared: ‘The Queen is here on the advice of ministers. She does not go out on a limb.’

The implication was clear — the fault for the accident-prone tour lay with Cook.

Obituaries of the peer who died on Monday — coincidentally the 43rd anniversary of Diana’s wedding to Prince Charles — focused on his integrity, sharp mind and rigour and, especially, the tight-rope he navigated as brother-in-law to Diana.

But there were missteps, not least in his relationship with Prince Charles. After Diana’s death, he was firmly of the view that the prince had to dump his then-mistress Mrs Camilla Parker Bowles if he ever hoped to take the throne.

Diana was a bridesmaid at Robert Fellowes’s 1978 wedding to her sister Jane

It was to Fellowes that Charles’s resolute rejection was made plain: Camilla, he said, was a ‘non-negotiable’ part of his life.

Figures from that time describe Fellowes as an ‘immense hindrance’ to the issue that was dividing Britain and threatening the monarchy.

‘The problem with Robert was that while superficially charming, he could not see any issue from any perspective other than the Queen’s,’ recalled one adviser.

‘A bigger brain might have allowed other views into the mix, but he never could. So problems just backed up and got worse, and it wasn’t until he left and Robin Janvrin took over — a much cleverer and more open-minded person — that healing started.’

Another insider recalled: ‘Robert was one of those who could never accept Camilla and who promoted the notion that Charles would have to finish with her if he was to make it to King. History shows how wrong he was.’

There were already tensions between the heir to the throne and his mother’s most senior official. Amid the furore over the 1994 Jonathan Dimbleby interview of the prince (in which he admitted his affair with Camilla), Sir Robert Fellowes — as he was then — was invited to check facts in the manuscript of Dimbleby’s biography of Charles for accuracy. But he was not shown the whole book, only those sections relating to the Queen and Duke of Edinburgh.

Passages which suggested that the prince blamed many of his woes on the coldness of his distant parents infuriated Fellowes, as this embarrassed the Queen.

He was opposed to the whole project and believed it was responsible for the PR disasters that followed — and particularly Diana’s own decision to go public in her now-infamous Panorama interview with Martin Bashir in 1995.

This was a particularly challenging moment for Fellowes as it forced him to put loyalty to the Crown over family ties. Despite Diana characterising him as one of ‘the enemy’, Fellowes never lost his affection for the young woman who was chief bridesmaid at his wedding to Lady Jane Spencer in 1978 — and wished to think the best of her, even then.

Robert and Lady Jane Fellows attend the wedding of William Duckworth-Chad – Prince William’s second cousin – and Lucy Greenwell in April 2011

When she called that November to tell him she had recorded an interview with the BBC one day in November, he innocently inquired: ‘Oh, Children In Need?’

‘No,’ the princess responded, ‘Panorama.’ Fellowes, she later said, was momentarily silenced.

For all his guile and silky skills, his time in the Queen’s office saw the popularity of the monarchy plunge from its Silver Jubilee peak in 1977 — the year he joined as assistant private secretary — to a trough from which some thought it might never climb out.

There were three divorces, the Queen had to give up the Royal Yacht Britannia, the Civil List was slashed, she began paying income tax and a dangerous rift opened up between Buckingham Palace and the Prince of Wales’s staff at St James’s Palace.

Yet there was another side to his occasionally rigid formality: a belief that the historical facts about the monarchy should be known to posterity. This was to make him a central figure in one of the most private controversies to rock the Palace.

It involved the historian Kenneth Rose’s 1983 biography of King George V, in which he disclosed that the Queen’s grandfather refused asylum to his cousin, the deposed Tsar of Russia, who was later murdered with his wife and children by the Bolsheviks. When a proof of the book arrived at the Palace, there was fury that the King was painted as a man with the blood of his own cousin on his hands.

At the time, Fellowes was assistant to the then private secretary Sir Philip Moore, who wanted the relevant passages removed.

Fellowes argued that as long as the facts were ‘incontrovertible’, they should remain. When the manuscript was put before the Queen, she wrote across the top, ‘Let him publish.’

With this, Fellowes, whose father Sir William ‘Billy’ Fellowes had been the Queen’s land agent at Sandringham, impressed her as a man of judgment.

In later years she understood his anguish when the crisis over the Waleses became public, as did the princess’s involvement in Andrew Morton’s sensational 1992 book, Diana: Her True Story, which lifted the lid on her unhappy marriage.

Three times he asked his sister-in-law if she had collaborated with Morton, and three times she denied it. ‘Certainly not,’ she answered categorically.

Fellowes passed on her denial to Lord McGregor, chairman of the Press Complaints Commission, who condemned the ‘odious exhibition of journalists dabbling their fingers in the stuff of other people’s souls’.

When, three days later, it became clear he had been deceived, a horrified Fellowes offered his resignation to the Queen. She turned it down, just as she had when off-the-cuff remarks that the Duchess of York had been ‘unsuitable for public life’ and her behaviour ‘foolish’ had emerged.

In subsequent memoirs, both Fergie and her father Major Ron Ferguson attacked courtiers, including Fellowes, who undermined her. Fellowes, whose mother Jane Ferguson was Major Ron’s aunt, was identified as ‘Mr Z’ in the duchess’s account. In private she called him ‘Bellows’.

But the rift with Diana that followed the publication of the Morton book was far more serious.

A family picture from the wedding of Princess Diana and Prince Charles at Buckingham Palace

Throughout the 1980s, Robert and his wife Jane had been the princess’s crutch. Close neighbours at Kensington Palace (the Felloweses lived in the Old Barracks next-door to Diana’s butler Paul Burrell) the sisters were constantly in and out of each other’s homes. William and Harry were also close to their three Fellowes first cousins.

Jane and her husband were sympathetic listeners as Diana poured out her unhappiness with Charles. They came to conclude that the Prince of Wales should have treated her better.

Inevitably because of his absolute loyalty to his royal employer, the relationship with his sister-in-law soured. However, when a wave of near hysteria swept the country in the wake of the princess’s death, Fellowes was among the loyal courtiers outraged by criticism that the Queen was somehow unfeeling by remaining in seclusion at Balmoral, instead of rushing back to London where there was mass mourning.

Nevertheless, he recognised there had to be visible concessions to public feeling. This led to the flag being lowered to half-mast over the Palace and the Queen brought forward her return to London before the funeral to meet the crowds outside her home.

He also wrote the speech the Queen delivered live from Buckingham Palace on the Six O’clock News on September 5.

Despite the omission of the word ‘love’ in the final draft it was a considered a tour de force thanks to the simple phrase that the monarch was speaking as ‘your Queen and as a grandmother’.

It soon emerged that the word ‘grandmother’ had been inserted by 10 Downing Street, a contribution New Labour aides have crowed about ever since. Yet it was the fastidious Fellowes who persuaded traditionalists that representatives from Diana’s charities should walk behind the gun carriage bearing her coffin, rather than the convention of soldiers and seamen.

Where Fellowes undoubtedly failed was allowing Tony Blair’s government to cream off much credit that belonged to the Royal Family.

Robert Fellowes had had no training for the ‘image battle’ between warring royals, and family members caught up in it, that was unleashed during his Palace years. As he once put it himself: ‘My life has been spent in quite a gilded cage.’

After Eton, he had taken a three-year short-service commission in the Scots Guards before becoming a discount broker in the City.

When he accepted the appointment to join the Palace in 1977, the Queen wittily remarked that he was ‘the first private secretary I have cradled in my arms’.

This was because Fellowes had been born on the Sandringham estate when the then Princess Elizabeth was 15. Indeed, Fellowes later shared a nursery with Diana (and his wife Jane’s) mother, Frances Shand Kydd.

At school he had been known as Bertie but that would never do at the Palace, where the Queen Mother pointed out that there was only one Bertie in the Royal Household — her late husband King George VI.

In a TV documentary marking the Queen’s 40 years on the throne, Fellowes was filmed receiving a crisp ‘you can come up now, Robert,’ on an intercom from the Queen at the start of the day, when he brought her post in a basket in one hand and a government box of official documents in the other.

After the Windsor Castle fire in 1992, he was at the forefront of discussion which saw the Queen pay tax on her private income following a public outcry at the proposal that taxpayers should foot the bill for its repair.

Constitutionalists said he should have fought harder to keep the Royal Yacht. ‘He let No 10 steamroller him,’ was the view in London clubs at the time.

Others, however, insisted that Fellowes was playing the long game. ‘He knew the public mood,’ recalled a friend. ‘His advice would have been to yield on this so she could fight the really big issues. Who knows, the state might have come after her palaces.’

However, in other areas of the Queen’s life, his handling was not so nimble. It was at his encouragement that she sat for the notoriously difficult artist Lucian Freud. The finished painting divided critics. Some said it made her look like a rugby prop-forward, others impertinently suggested she more closely resembled one of her corgis.

When one looks back at his life, his years at the Palace were, of course, dominated by the battles with and about Diana. He was acutely aware that his position and loyalty to the Queen caused his own wife Jane great pain. After the Panorama interview he advised the Queen that there was no alternative to divorce.

But following her death, it was Fellowes who mooted the idea of restoring Diana’s HRH style, which had been stripped from her.

He put the idea to his wife’s family, the Spencers, when they were en route to the private burial at Althorp with William and Harry, following the princess’s funeral. But by then it was too late and the gesture deemed inappropriate.

His relationship with the wayward princess was always complicated. But asked about Diana after her death, he said: ‘I was deeply fond of her. She was a very good person.

‘She found it difficult in life to find happiness, and I’m sad for people who have that situation.’