It's easy to forget what Memorial Day actually means while you're sitting by the pool and looking ahead at summer vacation—but the day signifies much more than just a three-day weekend.

Memorial Day is a solemn day of remembrance for everyone who has died serving in the American armed forces. The holiday, originally known as Decoration Day, started after the Civil War to honor the Union and Confederate dead.

It's unclear exactly where the holiday originated—Charleston, S.C., Waterloo, N.Y., Columbus, Ga. and other towns all claim to be the birthplace of the holiday. The event in Charleston that may have precipitated the holiday offers poignant evidence of a country struggling to rebuild itself after a bloody war: 257 Union soldiers died in prison in Charleston during the Civil War and were buried in unmarked graves, and the town's black residents organized a May Day ceremony in which they landscaped a burial ground to properly honor the soldiers.