Like many of the literary works produced in the late 19th to mid-20th century, Arthur Conan Doyle’s The Sign of the Four reflects the colonial project in a myriad of ways. Written a mere few decades after the 1857 Rebellion, and with a decidedly orientalist perspective, the work inculcates a specific breed of colonial anxiety prominent in the instability of Britain in the late 19th century. More specifically, this paper claims that the entry of Tonga and the Agra Treasure into London are events that precipitate British imperial anxiety within the narrative, and their respective arcs in the story – Tonga’s death and the loss of the treasure – are allegories for the ultimately unachievable desire to purge the colonial from the empire’s home bounds. This paper goes on to explore the imperial edifices and constructions developed within The Sign of the Four.