L’Enfant’s plan was only partially executed due to the shortage of materials and labor.

The finished construction had only two main stories, instead of the planned three. The porous sandstone walls were whitewashed with a mixture of lime, rice glue, casein, and lead. This achieved the white color of the building that later gave it its name.