If you’ve ever been to Assateague Island, chances are you’ve already seen this architectural piece of Eastern Shore history. Standing tall overlooking the Sinepuxent Bay and Assateague Island is the Flemish bond brick, monumental Rackliffe Plantation House—a watchful perch the home has maintained for nearly two hundred and seventy years. Imagine what this house has witnessed, and how many people have lived within her walls over the years.

The property itself was originally part of a 2200-acre land patent known as Genezeer, granted to Charles Rackliffe and his brother-in-law Edward Wale, who divided the acreage among them, making this land the oldest property owned by colonists in Northern Worcester County. For Eastern Shore natives from this part of the region, this land is where it all began. The bricks comprising Rackliffe House, and other properties of the same era just a few miles down the road—the slightly older Genesar, Fassit House, and several dwellings built a few decades later—serve as the very cornerstone for what would later become towns like Berlin, and much later, Ocean City.

The property Rackliffe House sits upon hardly began with the colonists.  Long before English settlers laid claim to the land, the Assateague Indians maintained a hunting camp on the grounds as evidenced by numerous artifacts—some are thought to date back as far as 10,000 years ago—found over the years and throughout extensive excavation. The Assateague Indians are a focal point for the Rackliffe House Trust, the foundation formed in 2004 that now maintains the property, after overseeing an extensive overhaul, costing nearly one million dollars.

Indian hunting camps, and colonial land grants were just the beginning in this property’s illustrious history. The Rackliffe House has witnessed the Revolutionary War, the War of 1812, and the Civil War.  In fact, a story of local folklore indicates that the second and third levels of the house may have been burned in an attack during the Revolutionary War. Well-known resident and local realtor Denise Milko, who lived in the property in the 1960’s recalls being told that a canon ball hit the house during the War of 1812.  As it turns out, the cannon ball story pales in comparison to Milko’s other tales from life on the former plantation. Like many very old houses, rumors of hauntings surrounding the Rackliffe House. These stories are made even more plausible by former residents like Milko who have their own firsthand accounts, and by historical records linking the ghostly tales to actual events. Tom Patton, Rackliffe descendent, and the greatest proponent of the Rackliffe House Renovation and Trust, beautifully described the history of the land surrounding the Rackliffe House in his book, Listen to the Voices, Follow the Trails. He also happened to call his ancestral home, “the most haunted house in the country.”

How History Becomes Haunted Legend – Who Those Rumored Ghosts May Be

Captain Charles Rackliffe reportedly built the current brick structure, in the 1740’s. (Earlier structures built of wood were likely part of the homestead, but have no current evidence of existence.) Captain Rackliffe used the now non-existent Sinepuxent Inlet to trade goods grown on the plantation with the British in exchange for bolsters of fabrics including fine silks and buttons. The Rackliffe’s kept detailed records. Inventories and wills from 1752, when Charles Rackliffe died, and in 1790 following the death of son John illustrate the vast holdings the family had—from fine furnishings to the plantation crop holdings, livestock, and even slaves.

It’s at this point in the house’s history, that the more infamous (i.e. haunted) stories take root. The Rackliffe’s were a family that enjoyed socializing; holding parties and dances routinely on the property, (possibly enhanced by the fact that a neighbor was well known for foraging through shipwrecks, often acquiring large amounts of liquor).  Former resident Milko recalls the property being known as “Sandy Point Dancing Floor,” because the floor of the great room, or ballroom used to have a bounce to the floorboards.

There was a darker side to plantation life. Slavery, though an unthinkable practice now, was one a commonplace, and provided the true backbone for a profitable plantation. According to local folklore, John Rackliffe was a particularly wicked slave owner, which ultimately led to his untimely demise. Court records substantiate that John Rackliffe was murdered at the hands of his slaves, who retaliated against him as he returned to the property late one night. Many years later, once Rackliffe House had acquired the reputation of being haunted, it was this story alone that could be linked back to documented fact—if such a thing could be a predictor of ghostly activity.

During the War of 1812, the British were said to have sailed along the Sinepuxent Bay, and while using the coastal homes as targets for cannon practice, also recruited colonial boys into service. A story of local lore regarding the Rackliffe house follows that a widow was living in the home during the war. When the British recruited her young son, she became so distraught, she hung herself in the third floor attic.

The third and final story involves what would have normally been a festive evening.  A lady, dressed for a ball tripped and fell down the stairs to her demise.  Who this woman was, remains a mystery. Could it have been Sara Rackliffe, wife of John, whose death very soon after her husband’s, left four young orphan children?  This type of speculation is the driving force behind folklore—the mixing of fact and theory that is the stuff of local legend.

The property stayed within the Rackliffe family, passing to relatives the Dirickson’s, until it was sold in 1939 at an estate sale, with portions of the land leading to the development of South Point.

Photo of the Rackliffe House taken in 2008 (by Robin Harrison) pre-renovations.

Beyond the Folklore—Real Life in the Rackliffe House, and Living With Ghosts

The infamous stories of the Rackliffe Plantation’s residents from the eighteenth and nineteenth centuries will always remain a part of the place’s most famous history. But, what was it like to live there during modern times, and is it really…haunted?

The day former resident Denise Milko walked back down the half-mile driveway, and back in time for this article, was overcast and had begun to rain.  Even amidst the graying weather, the property still loomed magnificent. Milko fondly recalls her time on the property, moving there in the 1960’s when she was seventeen years old.

At the time, a 20-gable barn and various outbuildings still stood.  Milko kept her horses there, and road all over the extensive property.  It was the horses that would first get riled up—indicating the night was about the get interesting.

Milko slept in an upstairs bedroom, and often woke up in the morning to her mother and various aunts discussing how loud she was all night—stomping, banging, and otherwise causing a ruckus in the room. When in fact, she had slept the entire night through—and heard nothing.

On other occasions, the piano downstairs would play by itself, soft footsteps were heard, an expensive smelling perfume wafted through the rooms, indicating (if the stories of former residents were true), perhaps the presence of the lady who had fallen down the stairs. A baby was often heard crying, loud noises kept the family up at night.

Houseguests unaccustomed to such otherworldly activity often fled the property, sometimes in the middle of the day, spooked from loud and unexplainable noises. In one case a cousin, who was a golf pro, heard a noise so loud it sounded like a piano crashing through the floors.

A particularly famous story happened during a family dinner party.  Prominent local residents were in attendance.  One spoke out loud that he didn’t believe in ghosts. The lights in the house immediately went out, and the candles flared, before the lights inexplicably being turned back on.

Of all the years Milko spent in the house, only one experience stands out as truly scary.  Alone in the house studying in her room, she heard the sound of a window shattering, followed by a gunshot.  Terrified, she called her father, and the sheriff.  Everyone would arrive at the home at the same time—to find nothing.  The windows and mirrors were all intact, and no one was found on the grounds. Yet no one questioned what Milko had heard—it was simply part of life on the property.

And that’s really the point with these stories—they are all part of the history of the place, a collective source of information that is more compelling than most fiction, legendary in folklore and spirits, and the seat of existence for so many.

Today, Rackliffe House has been meticulously restored. It is open during the summer months for tours, and by appointment and for special events. As for the ghosts…well perhaps they are pleased with the renovations. Find more information, here.

It’s worth noting that during the interview for this article, outside the plantation house, it had started to rain. When Denise Milko began to talk about the experiences she had in the house, specifically regarding the spirits, the rain stopped, the wind shifted and felt as though it was swirling around us. Coincidence, or something more?

Black and white photos from the Maryland Historical Society library.