The first reference to the tower was in 1366, in a sales document made between Bernat de So and Ponç de Descatllar for the bailiwick and castle governorship of Llívia. We do know however that the tower was built in 1584-1585 as part of a conflict with Puigcerdà over a long standing rivalry over the "recollita", an ancient Catalan right to gather the local population in times of danger.

Despite the economic and social crisis that took place in Llívia after the castle was destroyed, the town became prosperous again: the Llibre Ferrat was written, detailing all the rights of the town, and in 1528 Charles I granted it the title of "Villa", or town, which meant that Llívia remained part of Spain following the Treaty of the Pyrenees and the Treaty of Llívia. The tower was built between 1584 and 1585, together with the church, which was no longer used by 1617, and it became the new town defences.

At the end of the '50s, the Esteva family left the artefacts from the pharmacy to the Town Council, and in 1965 they were bought by the Provincial Government of Girona. This meant that in the '70s all the artefacts from the pharmacy were moved to the tower, until it became part of the collection in the Municipal Museum when it opened in 1981. From the early '80s until 2009, the top floor housed the offices of the Board of Trustees. After this date, the tower was used to exhibit slides on medicinal plants and today it is used for temporary exhibitions, starting with the scrolls from the Municipal Archives.