One hundred and forty years before Len Cullen opened Cullen Gardens, the property was actually the site of a busy and successful mill operation. In 1851, twenty three acres of the property was bought by James Mitchell, located on the east side of Lynde Creek. Here, Mitchell built Patly Mills (not Patley), which consisted of several buildings. Oatmeal and flour were milled at the site. James Mitchell died a tragic death in a shipwreck on Lake Ontario in 1853 and is buried at Oshawa Union Cemetery.
Eventually the mills and property were bought by J. Cooper. Cooper ran the mills from 1874 to 1878, and again from 1888 until his retirement in 1907 when the mills closed.
All that remains of Patly Mills today are: 2 stone foundations, the mill wheel, steel frames that held the mill pond chutes, the Mitchell foundations, the rotting wood of the Mitchell barn (along with a spinal column from one of his cows), the eastern foundation of the mill, remnants of the mill pond and of the mill pond's southern dirt walls. The Mitchell House was dismantled and now sits on the west side of the Cullen Garden complex with other historic homes from Whitby's past.
The mill was located exactly due east of what was once Cullen Gardens' Cottage Country section at the south end of the compound near Taunton Rd. Its foundation sit right beside the east side of Lynde Creek on a low lying section of ground that resembles a small island.
The historic mill stone was moved 150 metres north of where it once sat in the mills. It is located on a path around the north section of the old mill pond, which still has water in it..