James Figg was born in Thame in February 1684, the youngest of seven children born to Francis and Elizabeth Figg whom popular opinion believe lived in a row of cottages in Priestend. Being from a poor family in an agricultural town, his early life would not have been easy. He appears to have gained himself a reputation at prize fighting and is known to have frequented the booths at local fairs.