The first wooden manor on the site was constructed for Dukes of Masovia in the 15th century. It was later owned from 1516 by Anna Radziwiłł, Duchess regent of Masovia and Queen Bona Sforza after 1546 for whom an Italian style Renaissance garden was created. The new lavish wooden manor in mannerist style was built in 1570s for Anna Jagiellon. It was here that the premiere of blank-verse tragedy The Dismissal of the Greek Envoys by Jan Kochanowski took place on January 12, 1578. Sigismund III Vasa resided in the manor during the summer. Between 1602 and 1603, according to the Royal accounting books, the old manor was renovated and a new wooden house was built nearby. In 1606 the plan of the manor and garden was prepared for the king by Alessandro Albertini.
When in 1619 the king purchased the allotments belonging to Augustinian friars the construction of a new brick palace become possible. The spot for a Royal summer palace was chosen approximately 120 meters north from the original manor. According to the cornerstone founded in 1972 in the foundations of the eastern wing the construction started on September 16, 1624. The structure was designed by Matteo Castelli and Constantino Tencalla and accomplished after king's death by his son Ladislaus IV Vasa. In 1655 during the so-called Deluge of the Polish-Lithuanian Commonwealth (invasion of allied forces of Sweden from north, Brandenburg from west, Transilvania from south and Muscovy from East), the castle was devastated and remained practically uninhabited till 1668 when it was given to Teodor Denhoff.
It is a rectangular building with four octagonal towers at the corners, arcaded courtyard and a loggia with a view on Vistula River. Largely destroyed several times it was reconstructed in 1975.
Plan of the manor and garden in Ujazdów near Warsaw in 1606 by Alessandro Albertini, scale from. 1: 800, hand drawn multicolored document, 42 × 56 cm (16.5 × 22 in), signed: Il sito della villa di Jasdovia; Alessandro Albertini, 1606, Archiwum Główne Akt Dawnych w Warszawie, Zb. Kart. 570 – 1
Cornerstone of the Ujazdów Castle, sandstone, 57 × 57 × 10 cm (22.4 × 22.4 × 3.9 in), inscription in Latin: REGIAE AMOENITATI / SACRA / COELO SOLO LVCO LACV COLLE VALLE / LAETA / PALATIA AESTIVA / FELICIB[us]. FVNDAMENTIS AVSPICATVRV[m.] SAXVM / ANNO D[omi]NI MDCXXIV SEPTEMBR[e] / SIGISMVNDO III POLONIAE XXXVII / CO[n]STANCIA ANNO REGE / POSITVM / ANNO D[omi]NI 1624 DIE 7[septem]BRIS (Devoted Royal delight (...) summer palace), Muzeum Zamku i Szpitala Wojskowego na Ujazdowie