On the 28th June 1777 Andrew Bayntun Rolt married The Right Honourable Lady Maria Alicia Coventry, who was the eldest daughter of The Right Honourable, George William, the 6th Earl of Coventry. This marriage took place at her father's house in Piccadilly, in the Parish of St. George, Hanover Square and was performed by the Rev. St. Andrew St. John, clerk, a priest or member in holy orders of the Church of England, by virtue of a special licence granted for that purpose, by his Grace, the Archbishop of Canterbury.

The couple had first met and became acquainted a year earlier and were both 21 years of age at the time of their marriage. The witnesses who signed the marriage certificate were The Earl of Coventry, Sir Edward Bayntun Rolt (Andrew's father), Anne Coventry, William Harris and John Coventry.

Maria's mother, Lady Maria Coventry, was one of the Gunning sisters, of Castle Coote, Roscommon, Ireland, who became famous for their success in finding aristocratic husbands. She was a noted 18th century beauty and a lady of fashion. Ironically it was her beauty that eventually led to her death on 30th September 1760, when she suffered lead poisoning from cosmetic make-up. She was just 27 years old and it is said that over 10,000 people viewed her coffin. Print Collection, Miriam and Ira D. Wallach Division of Art, Prints and Photographs, The New York Public Library, Astor, Lenox and Tilden Foundations.

The affair was first noted at the beginning of September 1781 by Andrew's brother, the Rev. Henry Bayntun, when he visited the house on three separate occasions and saw the couple romping together and kissing eachother. Mary Nash, who was Lady Maria's personal maid and some of the other servants of the house were also aware of the affair and had seen them together on many occasions. Sir Andrew, at this time, was in London attending Parliament.

On the 9th December, Edward Baldwyn, one of the servants at Battle House, rode to Spye Park, the residence of Andrew's father – Sir Edward Bayntun Rolt and acquainted him of what he and the other servants had seen. Sir Edward asked Baldwyn to tell the same to the Rev. Henry Bayntun.

Andrew arrived home from London on Monday 10th December, and complying with Sir Edward's instructions, the Rev. Henry Bayntun, his brother, duly informed him of the affair. Later Henry was called to the drawing room at Battle House, as was Cooper, where Andrew immediately summoned and confronted his wife, who was pregnant at the time. He asked her to confess to the affair, which at first she denied, but eventually stated she had lain with Cooper on several occasions. Andrew asked her if the child she was carrying was his or Coopers. Upon learning it was not his, he asked for the return of her wedding ring and banished she and Cooper from his house. Andrew and Maria never laid eyes on eachother again from that day.

The following day, Maria sent her maid to personally deliver a letter to Andrew. In it, she begged his forgiveness and stated that although she had now become bitter enemies of both his mother and father, and in their eyes a wicked woman, she pleaded with him to believe that she would not be wicked enough to ever deceive him again and stated that she still held a glimmer of hope that he would be hers again.

But without a reply, she and Cooper, set off from the Greyhound Inn to his mother's house in Cumberwell where they lived together for a short while. Nothing further is known of the baby she was carrying when she left.

Divorce was rare at this time but on the 15th February 1783 it was granted to Andrew Bayntun Rolt by an Act of Parliament. Andrew was, at first, inconsolable and despite her shameless actions, he long lamented the mother of his only child.

However Cooper treated Maria with cruelty and brutality. The heartless destroyer of her life and fame, finished the dismal tragedy, shutting up her corpse in the house alone, until the rats had actually eaten part of her body. Yet this man was afterwards admitted to the best society and admired by all the ladies.

Death at least put an end to her sufferings and the young, elegant and accomplished Lady Maria, nurtured upon the bosom of indulgence, died in a low house, without a single friend or attendant to minister her last wants or a charitable hand to close her dying eyes. She died on the 18th January 1784 at the age of 29.

In 1787 Andrew married Anna Maria Maude, but this marriage failed to produce any children and he entered into a series of relationships that would eventually see him father many more children. Anna Maria outlived Andrew and died in 1827. Her will was proved at London on the 25th October 1827 and she had been living at Walcot in the County of Somerset prior to her death.

In 1788 Andrew's father, Sir Edward Bayntun Rolt, was forced to declare his granddaughter, Maria Barbara, first in the entail of the Bayntun estate when she was just 8 years old, which meant she would inherit her father's estate upon his death. The divorce meant that she would be Andrew's only legal and legitimate heiress.

Sir Andrew, in his early life, possessed very high moral qualities, but the misconduct of his first wife, to whom he was fondly attracted, altered his nature and he became a more carefree and reckless character.

He had two girlfriends – Harriet Maria Poynter who bore him 13 children and Ann Power who gave him another six – but many of these children died young. Andrew is said to have lavished a lot of money on his illegitimate children.

A tablet draped on a black slab in the Beauchamp Chapel in the Church of St. Nicholas in the village of Bromham reads: Here rest the remains of Sir Andrew Bayntun Rolt of Spye Park, Bart, only son of the late Sir Edward Bayntun Rolt, Bart. He married first Maria, daughter of George William Seventh Earl of Coventry and secondly Anna Maria Maude. He died 12th August 1816, aged 61, leaving one daughter by his first wife.

When Andrew Bayntun Rolt died the Baronetcy of one of Wiltshire's most famous families became extinct and his estate was inherited by his only legal heiress and daughter Maria Barbara Bayntun Rolt