From 1841 to 1921 Striblehills was home to the Castle family and the house was sometimes referred to as Castle Farm.
The Castle family were farmers and horse dealers of high-quality animals used for hunting and steeplechasing. They would have overseen all of the Victorian changes made to the house.

In 1865, William was elected as a Waywarden, responsible for the maintenance and repair of roads. He also served on the Local Board (the precursor to the UDC) and on the Board of Guardians, which oversaw the workhouse.
In 1867 William was fined for assaulting Henry Brazell. In 1881, his son George was attacked after pursuing a window breaker. On another occasion, barley was stolen from the family – clearly, they were not universally popular.