The Lodge of Unanimity building, at the junction of Sumner Road and St Davids Street in Lyttelton, was built in 1876-78 for the oldest Masonic Lodge to have been established in the South Island and the third to be formed in New Zealand. Less than a year after the arrival of the Canterbury Association immigrant 'First Four Ships' at Lyttelton, in October 1851, a meeting was held at the Canterbury Association's store in Lyttelton with the intention of forming a lodge under the Grand Lodge of England. An application was made through the Pacific Lodge in Wellington and was duly founded in May 1853 as the Freemasons Lodge of Unanimity No 879 (later to become 604 English Constitution, and eventually in 1890 it became Lodge of Unanimity No. 3, New Zealand). The foundation stone for the first lodge hall was laid in 1855 and the building was opened in October 1858. In 1876 that hall was demolished and the foundation stone for a new brick lodge building to the designs of Colonial Architect, Benjamin Woolfield Mountfort, was laid. It was opened in the same year but not completed until 1878 when the exterior was cemented and timber panelling was installed in the interior.

The design of the building is Victorian Free Gothic style, employing a simple interpretation of the Gothic Revival movement. The choice of such a style over the more common neo-classical design makes the building unusual for a lodge. The exterior has cemented walls and buttresses, parapeted gables and a steeply pitched slate roof. There is no fenestration in the original part of the building. In contrast to the relatively plain exterior, the interior is richly decorated and contains pointed arches in moulded relief wall design and an archived timber trussed ceiling. Masonic symbolism is found throughout the interior, including tessellated pavement (black and white squares) on the centre of the floor of the main hall, the square and compass, columns or pillars representing three ancient Greek orders of architecture, and tracing boards with symbols associated with Masonic principles of Truth, Love, Harmony, Peace and Charity.

In 1896-8 a timber refectory, also built in a restrained Gothic Revival style, was added to the original building to the designs of architect Cyril Mountfort, son of Benjamin W Mountfort. That addition has a large double window with timber architrave supported by decorative corbels. On the interior, the refectory walls having tongue and groove timber lining and the ceiling is of trussed timber of a simpler design than the original Lodge building ceiling.

Freemasonry has made an important contribution to the social, political and business life of New Zealand over the years. The Lodge of Unanimity has provided the setting for Freemasonry events since 1858, when the earlier building opened on the site. It helps inform about Lyttelton's history of having a relatively high number of Masonic lodges for the size of the town. Lodges provided a place where men could meet, exchange information and make social and business contacts. Lodge members have included a former mayor of Lyttelton (N C Schumacher), architect B W Mountfort and businessmen and local body members. Chattels of significance within the building include the Master's Chair made of English oak which was brought out to Lyttelton in the First Four Ships in 1850; Honours Boards of 1896 and 1976; the oak Altar, 1927; and Warden's columns which were presented by Fidelity No. 3 (English Constitution) in thanks for food parcels sent during World War Two and which are made from timber salvaged from the bombed out London Guild Hall.