The original house ‘Grittleton Manor’ which rested on the site was a three bay, Jacobean manor dating from 1660.

Major changes took place after Joseph Neeld aged 38 purchased the estate in 1828. Neeld was a London solicitor who had inherited a fortune of £900,000 from his uncle Philip Rundell (of Rundell & Bridge) the silversmith; Grittleton was to become his country seat. At that time the estate consisted of 1,248 acres within the parish. The present Grittleton House was built between 1835 and 1856, the architect being James Thomson. Later modifications were overseen by Henry Clutton.  Joseph Neeld was an enthusiastic patron of the arts and so built his palatial mansion to house his important collection of sculptures and other art treasures.

Sadly Joseph Neeld died before the present house was completed.   Joseph did not have a son to inherit the estate, so it passed to his brother John.  John Neeld not only completed Grittleton House, but also oversaw the completion of the houses in Alderton Road for the estate workers which had been started by Sir Joseph Neeld.  John Neeld died in 1899.  As John had six sons, it seemed safe to assume that the family name would endure.  All six sons however died without children.     

Sir Audley Dallas Neeld (last surviving son of John Neeld), died in 1941 at the age of 94 without heir.  As the Neeld family had no direct descendents, the estate passed to the illegitimate daughter of Sir Joseph Neeld, who in 1844 married Colonel William Inigo-Jones.  In 1942 Lionel Inigo-Jones succeeded to the property and changed his last name to Neeld and on his death in 1956 the property passed to his brother Ralph Inigo-Jones, known as Ralph Neeld.

In 1967 the Shipp family moved to Grittleton and rented Grittleton House from Ralph Neeld.  The Shipp family finally purchased Grittleton House in 1972.  Grittleton House is still owned by the Shipp family and is run by the family as an Independent Day School and is a fantastic venue for all types of occasions.