According to David Blight, professor of American History at Yale University, the first Memorial Day (formerly known as "Decoration Day") took place in Charleston, South Carolina on May 1, 1865. Organized by black South Carolinians and their white Northern abolitionist allies, it was the culmination of a series of events marking the end of the Civil War and the fall of Charleston.
In Race and Reunion: The Civil War in American Memory , Blight writes:
During the final year of the war, the Confederate command in the city had converted the planters' Race Course (horse-racing track) into a prison. Union soldiers were kept in terrible conditions in the interior of the track, without tents or other coverings. At least 257 died from exposure and disease and were hastily buried without coffins in unmarked graves behind the former judge's stand.
After the fall of the city, Charleston's blacks, many of whom had witnessed the suffering at the horse-track prison, insisted on proper burial of the Union dead. The symbolic power of the planter aristocracy's Race Course (where they had displayed their wealth, leisure, and influence) was not lost on the freedpeople. In conjunction with...the missionaries and teachers among three freedmen's relief associations at work in Charleston, blacks planned a May Day ceremony that A New York Times correspondent called "a procession of friends and mourners as South Carolina and the United States never saw before"...
During April, twenty-eight black men from one of the local churches built a suitable enclosure for the burial ground at the Race Course. In some ten days, they constructed a fence ten feet high, enclosing the burial ground, and landscaped the graves into neat rows. The wooden fence was whitewashed and an archway was built over the gate to the enclosure. On the arch, painted in black letters, the workmen inscribed "Martyrs of the Race Course"....
The war was over, and Memorial Day had been founded by African Americans in a ritual of remembrance and consecration. But the struggle to own the meaning of Memorial Day in particular, and of Civil War memory in general, had only begun.
(Sketch by A.R Ward, Harper's Weekly, May 18,1867)