The name of Penygawse (the Head of the Causeway) preserves the memory of the "cawse" which paved the bed of the Dyfri-Bawddwr at the point where it crossed the High Street, as it still did in 1675, according to Ogilby's map of the Presteign-Carmarthen road. In the eighteenth century, Penygawse was one of the properties of John Richards (1725-1803) who bequeathed all his houses to his only daughter, Ann (1758-1819), wife of John Jones (1757-1830). According to an indenture of 1854 among the deeds of the house, Ann devised "Penygawse where they lived" to her husband for life with remainder to their son, Isaac Jones. Ann's husband appears in the Rate Book for 1811 as "Mr. John Jones, Shopkeeper," occupying the "House and Garden." In the previous year the property had been valued on survey at 7s. 6d. a year, which was also the valuation of the Butter Hall of the time.

Ann Jones died on the l7th May, 1819, and on the 23rd September of that year her will, made on the 2ist September, 1814, was proved by her husband in the Consistory Court of St. David's. She was survived by her son, Isaac Jones, and three daughters: Jane, wife of Thomas Evans; Elizabeth Jones, spinster; and Maria, later married to Recs Jones.

John Jones, the father of these four, appears in Pigot's Directory of 1830, described as "Grocer and Draper," but while the directory was in the press he died, on the 5th April. The rate-books of 1836-1846 show the "House, Garden and Building" owned by Isaac Jones and occupied by his brother-in-law, Rees Jones, mercer. Directories of 1835-1844 describe Rees Jones as a grocer, draper and ironmonger. He appears annually in the Burgess Roll from its beginning in 1835 down to 1852, apparently still at Penygawse.

Of the children of John and Ann Jones, Jane Evans was the first to die, at a date unknown. Elizabeth Jones, spinster, died on the 23rd September, 1839. Maria, wife of Rees Jones, died on the 7th March, 1852, survived by her husband and their only son, John Jones. Isaac Jones died on the 16th January, 1853, survived by his widow, Mary Christian, and their two daughters: Ann Jones, spinster; and Elizabeth, wife of William George Warren. John Jones, son of Rees and Maria, died on the 25th June, 1853, aged only twenty-six years.

A sale of Penygawse took place on the 24th August, 1854, the parties named in the indenture being (i) Mary Christian Jones of Oakland Cottage, near Kingston, Jamaica; (2) Ann Jones, spinster, of the same, William George Warren of Belvedere Estate in the parish of St. Thomas in the East, Jamaica, and Elizabeth his wife; and (3) Rees Jones, late of Penygawse, draper, David Jeremy and William Jones Evans of Llandovery, gentlemen. David Jeremy was the manager of the Bank of the Black Ox and William Jones Evans was an attorney in Market Square.

Another indenture of the same date was made between David Jeremy and John James. The latter was the tenant of Penygawse, buying the property with the aid of the Bank. John James apparently became a householder in Llandovery about 1848, since his name was first entered on the Burgess Roll in 1851, implying a period of three years as a resident. In Hunt's 1849 Directory, Rees Jones and John James appear together as grocers and drapers in High Street. John James was also a stamp distributor - the operator of a sub-Post Office - and Slater's Directory of 1868 additionally describes him as agent for the Standard Life Insurance Company.

In 1880 John James lost his wife, Anne, and retired from business to live at Springfield Cottage in College View where he died in 1900. Although he had acquired the freehold of Penygawse by purchase, he was probably related to his predecessor, Rees Jones, since he and his wife and son are buried at Llandingat within the enclosing railings of the Penygawse lair, with John Richards and his descendants. The wife of John James may have been a daughter of Rees and Maria Jones.

When John James retired in 1880, the tenancy of Penygawse was taken by Morgan Griffiths who advertized his "Mantle and Millinery Rooms" in the Borough Guide in 1909, with a photograph of the house. After 1900, his landlord was John Rhys James, son of John James, who inherited his father's properties and came to live at Albert House (Butter Hall). He died in 1910 and was buried with his parents at Llandingat.