Standing out among the most important elements are the castle fortress, formed by the fortified nucleus and the keep, as well as the palace and the defensive walls. The castle dates from the 11th and 12th centuries, although it has subsequently been renovated. The defensive walls are from the 12th and 13th centuries, with the upper section reconstructed between the 16th and 18th centuries. The large moat that surrounds the town is striking, carved from the living rock, as Peratallada sits upon a clay rock base.
The two main squares in the town are Plaça de Les Voltes, a beautiful porticoed square, and Plaça del Castell, in front of the castle and the palace. Both are surrounded by small streets and very well preserved stone houses.
Beyond the walled area is the Sant Esteve parish church, of Romanesque style, which has two naves with semicircular apses, a portal under a voussoired semicircular arch and a four-eyed bell gable. Inside, it preserves a Gothic tomb from 1348.