The town was under the rule of the Venetian Republic from 1420 until 1797. The city's sixteen Baroque palaces were mostly built in this period, as well as its 17 Catholic churches and two Orthodox churches. The old city doesn't have a defensive wall, but instead it has nine defensive towers, the most important of which is the tower of the Holy Cross. These were built by the navy of the Venetian Republic in the 15th and 16th centuries.

At the fall of the "Serenissima" (1797) Perasto was the last city of the Republic to lower the Venetian flag. On 12 May 1797, the Republic of Venice ended, but a few places in the Venice for several months still continued to remain loyal to the Venetian Republic: Perasto was the last place of the Republic to surrender. On 22 August 1797 the Count Giuseppe Viscovich, Captain of Perasto lowered the Venetian flag of the Lion of Saint Mark pronouncing the farewell words in front of the crying people of the city and buried the "Gonfalon of Venice" under the altar of the main church of Perasto.

Established by the Benedictines, the abbey was first mentioned in 1166 in documents describing the consecration of the new, second Romanesque church of St. Tryphon in Kotor. The occasion was attended by, among others, Ivan, the abbot of St. George. However, in studying various ornamentation on this structure, it can be concluded that the abbey was already in use by the Benedictines as early as the 9th century.  Except for certain details, the appearance of the old church has not been preserved. The island was constantly under attack both by invaders and earthquakes, especially the great earthquake of 1667 when the ceiling and apse collapsed during the Easter service.  Following this catastrophe, a simple church was built. Its tombstones contain a unique collection of heraldic emblems from the casadas of Perast. It remained a burial place until 1866, when a new graveyard was built in the northern part of the town. The church once housed paintings dating from 1327 and 1457; the latter painted by Lovro Marinov Dobricevic, a famous painter from Kotor. 

The abbey of St. George had been under the jurisdiction of Kotor until 1634 when patronage of the island was transferred to the Senate of Venice.  In 1571, the pirate Karadoz burnt down the abbey as well as the whole town.  The people of Perast began repairs on the church in 1603.  In 1812, for about a year, the abbey was taken over by the French who were later expelled by the inhabitants of Perast from both the Fortress of St. Cross and the abbey.  In 1814, the abbey was taken over by the Austrians.

Our Lady of the Rock lies about 115 m (377 feet) northeast of the island of St. George. The stone plateau in the middle of the sea with a church on it is the result of the enduring efforts of generations of mariners “… to leave there, in the very heart of the Bay of Boka Kotorska, a testimony of their Christian faith and culture…”

The island was built artificially by scuttling old ships and depositing stones around a small crag.  The name of the island derived from the word skrpjel– an old word for a crag.  The construction of a sanctuary began in the first century of the Venetian rule.  The sanctuary was dedicated to the Assumption of the Virgin because this cult (Assunzione della Vergine) was greatly venerated by the Venetians.

 The original church was built in the mid-15th century and was of modest proportions. Over the centuries, the “island” was constantly enlarged and reinforced by both deposits of stones and scuttled sailing ships until it provided a sufficient base for a bolder architectural undertaking. Most of the present-day church was erected after the great earthquake of 1667 when the original sanctuary was destroyed.  It is a single-nave, modestly proportioned church in the Byzantine style. An octagonal 11m domed presbytery and a bell tower were added circa 1725. These gave the Our Lady of the Rock the distinctive baroque appearance that can be seen today. The interior of the church was decorated by Tripo Kokolja, a famous 17th-century painter from Perast. On the altar is the famous icon of Our Lady of the Rock, a 15th century work by Lovro Marinov Dobricevic.  It is the most valuable work of art in the church because its history is so closely related to that of the island.