In an era when wealthy American heiresses routinely traded their fortunes for European titles, one woman managed to do the opposite. Olive Paget married three times, divorced three times, and kept Leeds Castle in Kent through every single divorce. The medieval, moated fortress had once been home to six English queens. Olive, born in Manhattan in 1899, had inherited enough Standard Oil and Whitney family money to buy it, restore it at a cost of millions, and fill its weekend parties with guests ranging from Charlie Chaplin to the future King George VI.
Unlike nearly every other “dollar princess” of her age, Olive never allowed her wealth or property to fall into her husbands’ hands. Her fortune came from two sources, both rooted in the Gilded Age. Her maternal grandfather, William Collins Whitney, built his wealth through New York City infrastructure deals and served as Secretary of the Navy under President Grover Cleveland. He left an estate estimated at $23 million when he died in 1904.
Her second, far larger source of money was her great-uncle, Colonel Oliver Hazard Payne, brother of her grandmother Flora Whitney. Payne was a director and treasurer of Standard Oil whose assets exceeded $100 million. He never married and had no children, a fact that proved crucial to Olive’s future. Olive’s mother, Pauline Payne Whitney, was born in New York in 1874 and married Almeric Paget, an Englishman twelve years her senior, in 1895.
Almeric was the youngest of fourteen children of Lord Alfred Paget. He had impeccable lineage tracing back to the Marquess of Anglesey, who led British cavalry at Waterloo, but he inherited little of value. He remade himself from an Iowa cattle rancher into a Minnesota real estate man and eventually a New York society figure. The marriage moved to England in 1901.
Pauline inherited an additional $2. 5 million when her father died in 1904, and Almeric was elected as a Unionist MP for Cambridge in 1910. Pauline became a significant wartime philanthropist. In August 1914, she organized the Almeric Paget Massage Corps, recruiting trained masseuses to give physiotherapy to wounded soldiers.
She established a convalescent camp in Smarden, near Eastbourne, where wounded soldiers called her the “Angel of Smarden. ” She died in November 1916, aged just 42. Wounded soldiers she had helped carried her coffin. When Almeric remarried in 1921, it was to Edith Starr Miller, an American heiress who later published a book claiming a communist-Masonic conspiracy to control Europe.
Olive was born in Manhattan on September 24, 1899. Her mother’s chronic ill health moved the family to England when Olive was a small child, so she grew up in English country and London society while her fortune and extended American family remained in New York. Educated in France, she served briefly as a wartime nurse in 1918 near the end of World War I, following her mother’s medical charity work. Her mother’s death in 1916 made her independently wealthy as a teenager.
Sources differ on exact amounts: one account says $4 million was divided between the two daughters, another says roughly $5 million each. Both agree the money belonged entirely to Olive, entirely separate from any husband. Olive married three times. Her first husband was Charles John Frederick Winn, youngest son of the Second Baron St Oswald, whose family seat was Nostell Priory in Yorkshire.
They married in July 1919 and had two daughters, Pauline and Susan. The marriage collapsed in 1924 and divorce was finalized in 1925. The second marriage was the one that brought the castle. Olive married Arthur Wilson Filmer in May 1925, shortly after her divorce from Winn.
He was the son of an MP and grandson of shipping magnate Arthur Wilson. Using mostly Olive’s money, the couple bought Leeds Castle in 1926 for £180,000, roughly the equivalent of $7. 5 million today. The castle was in terrible condition.
It had sat empty for about two years, parts of the grounds were overgrown, there was no modern plumbing, lighting depended on oil lamps, and the servants’ quarters were described as a dark cellar. William Randolph Hearst considered buying it but backed out after his agent cabled that it needed immense expenditure and lacked bathrooms. Olive hired architect Owen Little for the initial restoration, then turned to French designer Armand-Albert Rateau, known for Art Deco interiors, who worked at Leeds from roughly 1927 to 1929. As her taste matured, she brought in Stéphane Boudin, head of the Paris decorating firm Maison Jansen, who worked at Leeds from the early 1930s until 1967.
Boudin’s other clients included the Duke and Duchess of Windsor and, most famously, Jacqueline Kennedy, whom he helped restore the White House in 1961. Beyond the interiors, Olive built tennis courts, a squash court, and a swimming pool with a wave-making machine imported from America. She converted the Maiden’s Tower into bachelor flats and a private cinema for weekend guests. At various stages the grounds featured llamas and zebras.
The total restoration cost from the 1920s to the 1960s has been estimated at about $2 million in the currency of the time, a figure repeated in sources without archival reference and best treated as a collective estimate rather than a verified total. Leeds Castle already had a rich history long before Olive bought it. The site had been occupied since at least 857 AD. Robert de Crèvecoeur rebuilt it in stone as a Norman fortress in 1119.
In 1278, Eleanor of Castile acquired it, and for over two centuries it formed part of the dower, the property guaranteed for life to successive English queens. Henry VIII converted it in 1519 as a residence for Catherine of Aragon. In 1552 the Crown granted it to Sir Anthony St Leger, and ownership eventually passed to the Culpeper family, whose over five million acres in Virginia created a transatlantic link with America. The castle passed by marriage to the Fairfax family and then to the Wykeham-Martin family, who rebuilt the main block in Tudor Gothic revival style in the 1820s.
Death duties forced the Wykeham-Martins to sell in 1924. Olive’s marriage to Wilson Filmer did not survive the restoration work. They divorced in late 1930 or 1931, and Olive kept full ownership of the castle. On November 4, 1931, she married Sir Adrian William Maxwell Baillie, the sixth baronet.
The New York Times covered the wedding under the headline “Mrs. Wilson Filmer Weds a Baronet,” noting the bride was the granddaughter of the late William C. Whitney and that this was her third marriage. Adrian had inherited the baronetcy as a teenager in 1914 after his elder brother was killed in the first weeks of World War I.
He had been educated at Eton and Sandhurst, served with the Royal Scots Greys in France, and worked in the diplomatic corps as second secretary at the British embassy in Washington from 1924 to 1928. His political career was turbulent, and he devoted himself to improving conditions for agricultural laborers. The couple had one child, a son named Gawaine, born in March 1934. During the 1930s, Leeds Castle became one of the greatest country houses in England.
The Baillies lived in London during the week and moved to Leeds for weekend parties, a classic English interwar social rhythm fueled by Olive’s American money and Hollywood connections. Guest lists are exceptionally well documented across multiple sources. Royal visitors included the Prince of Wales with Wallis Simpson, the Duke of York, Princess Marina, Queen Marie of Romania, exiled King Alfonso XIII of Spain, and Grand Duke Dmitri Pavlovich of Russia. Political guests included future Prime Minister Anthony Eden and, in a detail that later cast an uncomfortable shadow, Joachim von Ribbentrop, the German ambassador to Britain.
The Hollywood contingent included Douglas Fairbanks Sr. and Jr. , Charlie Chaplin, Errol Flynn, Frederic March, Robert Taylor, James Stewart, Gertrude Lawrence, Barbara Hutton, and author Ian Fleming. Contemporary and later commentary compared interwar Leeds Castle to something out of “The Great Gatsby,” with a BBC Radio 4 program calling it “the English Great Gatsby.
”
The golden era did not survive the decade. When World War II broke out, Leeds Castle was requisitioned for wartime medical use under a special agreement between Lady Baillie and the British Red Cross. The family moved to the back half of the castle known as the Gloriette. Lady Baillie, who had served as a VAD nurse in 1918, put on a uniform again in 1940, an honorary one this time, while her daughter Susan trained as a real nurse on the wards.
From July 1940, the 10th Company of the Royal Army Medical Corps occupied the castle. Throughout the Battle of Britain, the hospital treated victims of air warfare from both the British and German sides. The lake was drained in case its reflection attracted Luftwaffe aircraft. The castle’s most significant wartime connection was to the Guinea Pig Club, a mutual support and social network founded by pioneering plastic surgeon Archibald McIndoe at Queen Victoria Hospital in East Grinstead for British and Allied aircrew who underwent his experimental reconstructive surgery for severe burns.
A large part of the castle was given over to housing pilots recovering from surgery at McIndoe’s burns unit. Leeds Castle served as a convalescent annex to McIndoe’s surgical program, a country-house recovery environment where burned pilots could continue healing under the hospitality of Lady Baillie and her daughters. The distinction matters for accuracy: Leeds was the recovery site, not the surgical site, which remained at East Grinstead. The beauty of the castle itself was part of the therapy.
Hospitality resumed after the war on a smaller scale than in the 1930s. David Margesson and Geoffrey Lloyd remained permanent visitors and lifelong friends. Royal family members including the Queen Mother and Princess Marina continued to receive hospitality. John Hay Whitney, Olive’s American cousin, who served as United States ambassador to Britain from 1957 to 1961, brought renewed diplomatic presence to the castle’s social life.
Olive continued working with Boudin on interiors, completing plans interrupted by war. A new dining room and library were created in 1948. Her passion for exotic birds grew through the 1950s, with a systematic aviary-building program starting in 1957 that eventually produced a collection particularly distinguished by Australian parrots, cockatoos, and the rare blue-ringed parakeet. The fortune that supported all of this had been deliberately structured.
Whitney and Payne wealth reaching Olive through her mother was largely held in American trust arrangements that specified income beneficiaries and often isolated capital from a beneficiary’s spouse. This was a common feature of large American wealth of that era, designed partly as a convenient tax vehicle and partly as a deliberate protection mechanism against fortune-seeking husbands. In England, the Married Women’s Property Act of 1882 had already established that a married woman could hold property acquired before or during marriage as her own. The law was still relatively new when Olive first married in 1919, making her among the first generation of wealthy women to benefit fully from its protection.
Contemporary legal commentary confirms that assets settled in trust for one spouse’s separate benefit are generally protected from division upon divorce in a way that joint or mixed marital property is not. The contrast with the typical dollar princess fate is striking. Consuelo Vanderbilt’s marriage settlement with the Duke of Marlborough handed over $2. 5 million in railroad stock and a guaranteed annual income of $100,000 directly under her husband’s control as part of the marriage contract itself.
Olive’s guardians, or more precisely the family lawyers and trustees managing the Whitney Payne money on her behalf, achieved the opposite. Olive got the titles, got the castle, and kept the money because the legal structure of her wealth had been designed a generation earlier to ensure no husband could ever reach it. The European estate was absorbed into her American fortune, and the European husbands were absorbed into her social world and then expelled from it when the marriage ended, without taking a single room with them. Olive never married again.
For the remaining thirty years of her life, she kept the castle entirely on her own terms, answerable to no husband and bound by no marriage settlement. Her health declined steadily from the 1950s onward as a result of lifelong heavy cigarette smoking, which had been constant in every decade of her adult life. By the early 1970s, she needed supplemental oxygen and a resident nurse. She had amassed large collections of French and English furniture, furnishings, and paintings over four decades.
Her sister Dorothy Paget, who inherited a similar fortune, became one of the most successful racehorse owners in British racing history, winning the Champion Hurdle five times and the Cheltenham Gold Cup four times. Her cousin Gertrude Vanderbilt Whitney founded the Whitney Museum of American Art. The family money could fund very different obsessions across the same generation. Olive died in London on September 9, 1974, fifteen days before her 75th birthday.
Her estate was valued at approximately £4. 08 million. The most important decision of her later life was her explicit refusal to allow Leeds Castle to pass to the National Trust, the traditional destination for great English country houses whose families could no longer afford them. Instead, she arranged for the castle to be administered by a charity of her own creation, the Leeds Castle Foundation, formally registered as a charity on October 29, 1974, just seven weeks after her death.
The foundation’s declared purpose was to preserve the castle, its collections, and its grounds for the benefit of the public. A secondary objective reflected Olive’s interests and the castle’s wartime medical history: providing a place for discussion and debate on medical science and a forum for the arts. Geoffrey Lloyd, Olive’s close friend since her 1930s party days, served as the foundation’s first chairman. The foundation’s medical science mandate reflected Olive’s wartime experience and the castle’s Guinea Pig Club connection; Leeds Castle hosted international medical conferences in the decades that followed, including events linked to the discovery of the polio vaccine and developments in burn treatment.
The foundation quickly realized it needed commercial income. Since then, Leeds Castle has operated as both a tourist destination and a conference venue. The foundation opened the grounds to the public in 1975 and the castle itself in 1976. By the 2020s, it was recording annual income of around £12 million and employing more than 200 staff.
Olive’s arrangement deliberately separated the castle building and its contents, which she bequeathed to the foundation, from the surrounding land, which she had given to her son Gawaine in 1966. The family kept the land while the castle itself became public property. The land surrounding the castle remains owned by the Baillie family to this day, creating a permanent division between the public castle and private land, a structure that has persisted for half a century. Gawaine Baillie died in 2003.
It was then revealed that he had secretly assembled one of the greatest stamp collections of the former British Empire ever put together. Sotheby’s sold it across ten auctions that collectively exceeded initial estimates by more than £4 million. The secrecy of the collection was itself a family trait. Gawaine told almost no one about the stamps, just as his mother told no one about the legal arrangements that kept the castle in her name through three divorces.
Three husbands came and went, and the castle survived them all. The woman who owned it learned from the generation before her that the only wealth a woman could trust was wealth no husband could touch. The foundation she created in her final years ensured that the lesson she learned would protect the castle long after the last marriage, the last divorce, and the last trust deed signature had faded into archive.