After meeting up with Dr Bruno Werz in the parking lot of Cape Town’s Dolphin Beach Hotel, we picked our way through tussocked dunes down to the Table Bay shoreline. As tourists snapped selfies in front of the unmistakable backdrop of Table Mountain and kite surfers whizzed past us in the direction of Blouberg Beach, Werz pointed out a choppy spot some 60m from the shore.

“I’m 95% sure the wreck is just there,” he said. “Which means the survivors’ camp is buried somewhere in the dunes behind us.”

On 25 March 1647 – five years before the Dutch East India Company, Vereenigde Oostindische Compagnie (VOC) in Dutch, had established the Cape Town settlement just north of the Cape of Good Hope – the Nieuw Haarlem foundered in Table Bay’s shallow waters. Luckily, no lives were lost, and much of the precious cargo the ship was bringing back to the Netherlands (via South Africa) from Asia was salvageable. Not long after the incident, 58 crew members were taken back to the Netherlands by the other ships in the fleet. But the remaining 62 men were left behind to look after the valuable spices, pepper, textiles and porcelain until a larger fleet could give them and their cargo a lift home about a year later.

If they hadn’t stayed, said Gerald Groenewald of the University of Johannesburg’s Department of History, “the history of colonial South Africa could have turned out very differently.”

You may also be interested in:
• Cape Town’s slave ship secret
• A shipwreck worth billions
• The mystery of Cape Town’s disappearing gun

Dutch and other European ships had been stopping at Table Bay and Saldanha Bay (some 130km to the north) since the 1590s to load up on drinking water and barter livestock. But the experience of the Nieuw Haarlem survivors was the “catalyst” that determined which of the powers would be the first to settle in the region and where precisely they would settle. For the Dutch, it was Cape Town. After 1652, according to Groenewald, “the English started to concentrate more on St Helena as a halfway station. The French continued to call at Saldanha Bay from time to time but also had their own colony in Reunion.”

Werz, who started his career as a marine archaeologist in the Netherlands, moved to South Africa in 1988 to take up a lecturing position at the University of Cape Town. Within a few weeks of arriving, a member of the public phoned to say she thought she’d found the remains of the Nieuw Haarlem.

Werz had come to South Africa harbouring hopes of discovering the remains of the many VOC ships wrecked at the Cape. The Haarlem – the ship that “started it all” – was at the top of his wish list. It was a mystery that many serious historians had wanted to solve over the years, and previous attempts to find the wreck had all included many incorrect assumptions.

“She took me to the beach and showed me some timbers,” said Werz, who instantly recognised them as coming from a much more recent 19th-Century wreck. Putting the disappointment behind him, he continued to investigate the Nieuw Haarlem “as a sort of hobby,” by walking on the beach “whenever [he] was bored” and combing the state archives in the Hague and Cape Town for any information. Through his research, he found a tranche of documents, including the journal kept the by the ship’s junior merchant, Leendert Jansz, who was one of the 62 men who had stayed in Cape Town after the wreck. The journal provided Werz with great detail about the year the men spent at the Cape – including a few very useful clues as to the exact location of the wreck.

After the men had made it ashore in 1647 – transferring heavy cargo in the choppy conditions was an ordeal which spanned several weeks and claimed one life – they established a camp among the dunes which they called Zandenburch (“Sandcastle” in English). They bartered livestock and fresh meat from the indigenous KhoeKhoe people and had great success fishing in the nearby Salt River. They even ventured to Robben Island – where Nelson Mandela languished in prison from 1964 to 1982 – in skiffs. (The island is now a popular tourist attraction with a museum, where visitors can learn all about the prison and see the resident penguins which are now an endangered species.)

This excerpt from Jansz’s journal gives an idea of the kind of challenges and successes the men experienced. On Saturday, 15 June, “the crew shot a rhinoceros (which had been fighting with an elephant) near our fortification. The meat is very short-grained and tastes good, which serves us very well in this time of shortage.” And a day later, “our skiff returned from the Robben Island with 200 birds, mostly penguins, as well as 800 eggs.”

Despite the hardships, Jansz was able to recognise the potential of the Cape as a victualling station for Dutch ships – he just had to convince the Lords Seventeen (the VOC’s “board of directors”), who were notoriously stingy, according to Groenewald, to establish a permanent settlement there.

Taking on fresh provisions was vital to the wellbeing of crews making the eight-month, one-way journey to and from Asia, with scurvy and other diseases claiming many lives. VOC ships had a history of calling at various places in the southern hemisphere, including St Helena, Mauritius and Table Bay; however, they were hesitant to establish a presence on the African mainland because of the perceived costs of such an endeavour and because they were wary of the people who already lived there.

So, after returning to Holland in 1648, Jansz and another official named Proot (whose identity is in doubt but is most likely Matthijs Proot, another junior merchant who was not on the Haarlem) submitted a remonstration to the Lords Seventeen that “succinctly demonstrated and indicated what service, advantage and profit will accrue to the VOC by constructing a fort and garden at the Cape of Good Hope.”

The document pointed out the Cape’s strategic location (if enemy ships wanted to attack, it noted, there would be “no better place in the world for them to do so than the Cape”), fertile land, abundant fish and livestock, and access to fresh water and timber. Most importantly, the remonstration stressed the fact that “the indigenous people came in all friendliness to barter, [bringing] cattle and sheep in quantities.”

The remonstration went to great pains to convince the directors of how cheaply the settlement could be established. It also dismissed the popular conception at the time that indigenous Africans were “man-eating people” as “nonsense”, explaining that “the killing of our people is caused more by revenge due to the stealing of their cattle, than because they want to eat us.” It also recommended (with remarkable prescience) that the new settlement should have “a good commander who treats the indigenous people politely and who pays for everything that is bartered from them, and to treat some of them with a bellyful of food.”

Instead, Jan van Riebeeck – an ambitious VOC official who was on one of the ships that collected the last of the Nieuw Haarlem survivors in 1648 – successfully petitioned to be the Cape’s first commander and took things in a different but history-defining direction. “From day one, he had a negative view of the KhoeKhoe,” said Groenewald. “He had a low opinion of them and distrusted them very much. His ill relationship with the KhoeKhoe led to the first KhoeKhoe-Dutch War of 1658-59.”

In the decades that followed, the initial vision of a contained outpost at the Cape was obliterated, particularly when van Riebeeck made the decision to allow the Dutch to farm in the hinterland. While the first Free Burghers (“free citizens” who were released from VOC employment to farm their own land) originally settled near to Cape Town, as their numbers swelled, they ventured further and further inland, expanding the footprint of the settlement. And, in 1658, the first slaves were imported by the VOC (initially from West Africa and later from areas around the Indian Ocean) to do hard labour.

This “socio-economic cocktail”, said Jane Carruthers, an emeritus professor at the University of South Africa, laid the foundation for the “social order and class distinction based on race”, which was cemented in the 20th Century.

According to Werz, finding the Haarlem would “make one of the most important pieces of 17th-Century South African history tangible.” By 2015, the project had become much more than just a hobby and he had gathered folders of information on the wreck. Jansz’s journal mentioned that the ship was sunk exactly 1.5 miles from the place where van Riebeeck would eventually establish his fort. While those who’d looked for the Nieuw Haarlem before him had assumed the figure to be in English miles, Werz knew that a Dutch mile was almost five times longer than an English mile.

Another clue came from a 1652 volume which stated that a drinking well dug at the survivors’ camp was “60 feet deep, dug through sand, then limestone, then shelly sand, then clay and, at the deepest level, sand with water.” Such information (known as a stratigraphic column by geologists) served as a unique “location barcode” that Keletso Mulele, one of Werz’s honours students, compared to a 1970s geological survey to further zero in on the location of the survivor camp and the nearby wreck.

In 2016, Werz teamed up with geophysicist Billy Steenkamp to use a magnetometer (a type of metal detector that only picks up iron) to search the area he’d earmarked. Excavations on the beach yielded the remains of a 19th-Century shipwreck, not to mention fencing poles, a piece of pipe and a marine engine block. When they ventured a bit deeper, however, their luck began to change. They were able to dig up a huge semi-circular object that Werz believes was some sort of clamp from a mast or spar and a number of hand-forged nails which could only be very old. They even found a necklace made from copper, which Werz thinks was crafted by the KhoeKhoe people, possibly bartered from the Nieuw Haarlem survivors.

While it must have been tempting to continue using earth-moving equipment to solve the mystery once and for all, Werz called the dig off when a metre-long nail was snapped in half by a digger. Unveiling the last of the Nieuw Haarlem’s secrets will require erecting a temporary coffer dam and draining all the water from the area. Once he’s secured funding for this final phase of the project, Werz said, “the ultimate proof will be the discovery of the 19 iron cannon and four iron anchors that were left behind in the hold.”

Werz has been in the game long enough to know that this moment is still quite a long way off, but he is hopeful that further evidence (he’s waiting for metallurgical reports on the finds and he hopes to drill the site for wood samples soon) will lead to the erection of some kind of monument at the wreck site. For now, visitors will have to park at the hotel and walk along the shoreline in the direction of the city for about 10 minutes – and use their imaginations to envision the camp and wreck site.

And, if and when the cannon and anchors are found, they would certainly warrant the creation of a museum display. “I know of no other shipwreck anywhere in the world,” he stressed, “that has had such an impact on an entire nation.”

Sunken Civilisation is a BBC Travel series that explores mythical underwater worlds that seem too fantastical to exist today but are astonishingly real.

Join more than three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called “The Essential List”. A handpicked selection of stories from BBC Future, Culture, Worklife and Travel, delivered to your inbox every Friday.