He became famous in London society for his affair with the future Prime Minister's wife, Caroline Lamb, and for his love of his half-sister, Augusta Leigh, who left her husband in 1811 and gave birth to a daughter in 1814. Byron's joy at the child's birth sparked speculation about his paternity.
To avoid scandal, Augusta persuaded Byron to "marry for convenience". He reluctantly agreed to marry Anna Isabella (Annabella) Milbanke, a cousin of Lady Caroline Lamb. A daughter, Ada, was born of this marriage.