The house was built for General Robert E. Lee when he was chosen as president of W&L after the end of the Civil War. Since then it has served as the house of the President of the University. Research has shown it to be not an original design, but one adapted from a plan book of the time. The large porch, with its low, almost transparent railings, was for the benefit of Martha Custis Lee (Lee's wife, a great-granddaughter of Martha Washington), who was confined to a wheelchair. She is said to have enjoyed watching and talking with students from her vantage point on the porch.
Lee died in 1870, at 63, in the dining room of this house.