Located in from of La Caleta Beach, the castle was ordered to be built by King Philip II in 1598, after the English assault on the city.
The project is that of the military engineer Cristóbal de Rojas, who died in the city before construction was finished. The project was more ambitious than what was ultimately built, at the forefront of the fortress a bastioned front was planned on being built, that next to the castle would become a citadel of the city.
Of pentagonal plan, the part that faces the sea displays a star with three ends. The part on land has a wall with a door in the center, flanked by two half-bastions and moat with a stable and drawbridge. The rake of this door was reused as the Arch of the Pópulo, then in disuse.
In 1693, reigning Charles II built the chapel and the sacristy, with a single nave covered with a barrel vault and high choir at the bottom.
The entrance, very sober, presents an overhang supported by pilasters and a small belfry; Inside there is a single altar with a baroque altarpiece that has lost the gold. In it are venerated the images of St. Catherine, St. Joseph, St. Augustine, St. Francis and others.
In the interior there are pavilions, barracks and cisterns, originally of powder. Later it became a military prison. It has been restored and the waterproofing of the roofs and walls has been proceeded; The interior vaults of the rooms and the stone columns of the exterior porch have been recovered, which resembles a part of a main square of a Castilian city. In addition, a stone from Tarifa has been used as flooring of this enclosure in the open air; The great entrance gate to the castle was recovered, as well as the old punishment cells, which have become a space dedicated to craftwork.
It consists of three exhibition halls that alternate their contents, and a room with a permanent exhibition about the sea.