The castle of Sommerhausen is located on the other side of the town hall. It was owned by the countess Hildegard von Rechteren-Limpurg. When she died it passed into other hands and was gradually renovated. Nowadays it used for various events. The monumental three-storey gable with its pinnacle-like attachments dominates the townscape.
The main building dates from 1569, whereas the older part, which consists of two wings, goes back to the 15th century. The former owners, the counts of Rechteren-Limpurg, used to hold the office of the Imperial Hereditary Cupbearer at the Imperial Court (cf. Ludwig Uhland’s poem “Der Schenk von Limpurg”). In the course of the Thirty Years’ War, Emperor Ferdinand III took up quarters in the castle which was plundered by Piccolomini’s troops in 1634. Gustav Adolf’s march through Sommerhausen is shown on a mural painting at the façade of the hotel “Ritter Jörg”. The picture was created by the above-mentioned artist Irmingard von Freyberg.