Westoe was built for Sir William Fox (1812? -1893). Fox, and his wife Sarah, arrived in New Zealand in 1842. In 1843 Fox, who had trained as a lawyer in England, was appointed the New Zealand Company Agent to Nelson and eventually, Principal Agent for the company. Over the next thirty years he was to influence New Zealand political life, first as a Member of the Wellington Provincial Council, and then as a Member of Parliament. During his varied public career Fox was to hold the position of Colonial Secretary, Attorney General, and was four times the Premier. In later life Fox was to hold the position of the Commissioner of Land Claims arising from the confiscation of Maori lands on the West Coast and Taranaki. Fox was also a talented painter, explorer, and keen advocate for the temperance movement which aimed to reduce alcohol consumption.

Fox acquired 5000 acres of the original Rangitikei block in about 1849 (confirmed by a Crown grant in 1858). Fox was later to subdivide a portion of this land for the proposed town of Marsden. The settlement proved unpopular due to Fox's stipulation that liquor should not be sold from any of the buildings erected there. The early Pakeha settlers eventually chose to settle further north at what is now the town of Marton. Fox named his own property Westoe after the town in Durham County, England, where he was born. He built his first house in the early 1850s on river flats. Unfortunately the location proved to be prone to flooding and, when the opportunity arose to build a new house, the decision was made to locate it on the flat above the original homestead. Designed by Charles Tringham, the new house was built in 1874, and the carpenter was probably H. Austin of Halcombe. It is said that Tringham was inspired by Osborne House, the home of Queen Victoria on the Isle of Wright, and parallels can be seen with the Italianate tower which is a central feature of the house. The Westoe tower was never completed and remains unlined inside. Fox was a keen amateur horticulturist and created a substantial park and garden surrounding Westoe. Fox sold Westoe in 1885 and moved to Auckland. A few months later the house and lands were acquired by James Howard.

Westoe, in its splendid setting, is significant as the house of William Fox, a noted politician, artist, and social advocate, and is a fine example of a Victorian country house.