Prince HarryandMeghan Markleare sharing new details aboutthe Sandringham Summit.

“I sent an email to the three most senior private secretaries saying, ‘Let’s have a meeting, let’s get together and have a meeting and talk about this,’ " Harry said. “Because what was happening, what was playing out in public was crazy. And that meeting was rejected.”

For more on the Royals, listen below to our daily podcast PEOPLE Every Day.

“It was only once Meg had left and gone back to Canada that it was then arranged that there was going to be a meeting at Sandringham on the following Monday,” he continued.

“Imagine a conversation, a roundtable discussion about the future of your life,” said Meghan. “When the stakes are this high. And you as the mom and the wife and the target, in many regards, aren’t invited to have a seat at the table.”

Chris Jackson/Getty

Prince William, Prince of Wales, Prince Harry, Duke of Sussex

“It was clear to me that they planned out so that you weren’t in the room,” added Harry.

Prince Harry then revealed the inside events of his approximately 90-minute discussion with Queen Elizabeth, the then-Prince CharlesandPrince Williamat the monarch’s country estate in Norfolk to “talk things through” on Jan. 13, 2020 — five days after Meghan and Harry released their statement.

Prince William and Prince Harry.Alamy Stock Photo

The Duke of Cambridge and Duke of Sussex

“I have suchhappy memories of Sandringham— it was where we spent every Christmas — and now I was back there in very different circumstances,” he said about his fondness for the royal family’s special annual gathering. During her life,Queen Elizabethwould travel to Norfolk in mid-December and stay there until the anniversary of her father’s death and her accession to the throne in early February.

“I went in with the same proposal that we’d already made publicly. But once I got there, I was given five options,” added Harry. “One being ‘all in, no change,’ five being ‘all out’.”

“I chose option three in the meeting,” he said. “Half in, half out; have our own jobs but also work in support of the Queen.”

Despite this, he said that his decision provoked a reaction from the family that was not one of compromise — and led to his older brother,Prince William, shouting at him.

“It became very clear very quickly that that goal was not up for discussion or debate,” said Harry. “It was terrifying to have my brother scream and shout at me, and my father say things that simply weren’t true and my grandmother quietly sit there and sort of take it all in.”

Sandringham House.Indigo/Getty

Sandringham House

According toPrince Harry, the meeting “finished without any solidified action plan.”

“I think from their perspective, they had to believe that it was more about us and maybe the issues that we had, as opposed to their partner, the media, and themselves. That relationship that was causing so much pain for us. They saw what they wanted to see,” the Duke of Sussex said.

“The saddest part of it was this wedge created between myself and my brother, so that he’s now on the institution’s side,” he continued. “Part of that I get, I understand, right, that’s his inheritance. So to some extent, it’s already ingrained in him that part of his responsibility is the survivability and continuation of this institution.”

Prince William and Prince Harry.Chris Jackson/Getty Images

Prince William, Prince of Wales and Prince Harry, Duke of Sussex walk behind the coffin during the procession for the Lying-in State of Queen Elizabeth II

“We couldn’t believe it,” the Duke of Sussex said. “No one had asked me permission to put my name to a statement like that. And I rang M and I told her, and she burst into floods of tears because within four hours, they’re happy to lie to protect my brother. And yet, for three years, they were never willing to tell the truth to protect us.”

Meghan said in the docuseries, “Suddenly, what clicked in my head was ‘it’s never going to stop.’ Every rumor, every negative thing, every lie, everything that I knew wasn’t true and that the palace knew wasn’t true and internally they knew wasn’t true, that was just being allowed to fester.”

Prince Harry, Duke of Sussex, Meghan, Duchess of Sussex and Queen Elizabeth II

Can’t get enough ofPEOPLE’s Royals coverage?Sign up for our free Royals newsletterto get the latest updates onKate Middleton,Meghan Markleand more!

Shortly after the Sandringham Summit,Queen Elizabethreleased a rare and emotional statement.

“Today my family had very constructive discussions on the future of my grandson and his family,” she said. “My family and I are entirely supportive of Harry and Meghan’s desire to create a new life as a young family,” she continued. “Although we would have preferred them to remain full-time working Members of the Royal Family, we respect and understand their wish to live a more independent life as a family while remaining a valued part of my family.”

“Harry and Meghan have made clear that they do not want to be reliant on public funds in their new lives. It has therefore been agreed that there will be a period of transition in which the Sussexes will spend time in Canada and the U.K.,” she continued. “These are complex matters for my family to resolve, and there is some more work to be done, but I have asked for final decisions to be reached in the coming days.”

In March 2020,Prince Harry and Meghan took part in their final round of royal outingsbefore relocating to Meghan’s home state of California. Almost a year later, in February 2021, Buckingham Palace confirmed that the couplewould not be returning to their royal roles.

source: people.com