The origins of the collection are connected with the person of Prince Augustus Wettin, who often bought works of art and craftsmanship, but his attention was mainly focused on applied art and armaments. This changed during the reigns of Augustus II the Strong and his son, Augustus III of Poland, who, with large financial resources and connections at European courts, purchased paintings by European masters on a large scale. Their collections were exhibited in various buildings, eventually ending up on permanent display in Zwinger, the Semper Gallery.
During World War II, the paintings were hidden outside the Sempergalerie, where they survived Allied air raids. Later they were taken to the Soviet Union. In the 1950s, most of the collection was returned, although several hundred works are still missing.
