Wavertree Mill is said to have ceased regular operation in 1873, but seems to have worked intermittently until 1890, when the final lease from the Marquess of Salisbury expired. Severe gales in 1895 wrecked the sails and damaged the structure beyond repair, but it was 1916 before the decaying skeleton was finally pulled down. After that, the area became the site of lock-up garages; laid out in a crescent (backing on to the old public footpath) so as to leave the mill's foundations exposed. After nearly seventy more years as a local talking-point, regularly visited by groups of schoolchildren studying local history, the site was finally obliterated in 1986.

During the Victorian era, Wavertree Mill was widely regarded as being jinxed. In July 1866, ten-year-old Richard Matthews died after being struck by the sails of the windmill. His father (also called Richard) was Col. James Bourne's coachman. Some years later, the sails caught the hair of Charles Taylor's eldest daughter. The local historian James Hoult tells us that "she was scalped and was rendered insensible for twelve hours, but happily she recovered". Rumour spread that the old quarry was inhabited by an evil spirit. Eventually, it is said, the miller took the advice he was given, and set the sails in such a way as their shadow formed a cross touching all four corners of the quarry. No more accidents are recorded after that!