Dressed in a Union Civil War junior officer’s frock coat with epaulets and a purple-felt top hat displaying an array of long feathers in red, green, yellow and blue, Joseph Amster leads a group of eight – men, women and teenagers – across Powell Street near San Francisco’s Union Square.

With his grey hair and circle beard, Amster already draws attention for his unique look, though it’s the confident way he struts – puffing out his chest with his chin held high and waving a walking stick at arm’s length like a baton – that really attracts onlookers.

Some people seem fascinated by his presence; others look bewildered. But it’s the many passers-by who don’t even give Amster a second glance that truly encapsulate the city’s reputed tolerance for the outlandish and absurd.

Northern California’s cultural hub, San Francisco, has always been known as a city of nonconformity. It’s a place where residents paint their Victorian-style homes in pastel hues or the entire spectrum of the rainbow; run May’s annual Bay to Breakers race dressed as bacon slices or unicorns, or, at times, completely nude; and are just as comfortable dining out in leggings and flannel as they are in thigh-high boots and sweaters.

But in recent decades, a deluge of tech workers lured through the financial promises of the nearby Silicon Valley have challenged San Francisco’s free-spirited nature, inflating already-outlandish San Francisco rents and everyday prices and pushing living costs to 25% higher than the national average. These high fees are driving out many of the city’s artists, writers and long-time residents; while the housing shortage is also partly responsible for the increasing number of homeless citizens, many of whom work regular jobs but still can’t get a leg up in this city that caters increasingly to the elite. With so much disparity, many wonder whether San Francisco’s progressive, open-minded attitude can survive.

You may also be interested in:
• Why the French love to say no
• The secret to Danish happiness?
• Why San Francisco does sourdough best

“San Francisco has been a city of change since day one,” Amster, a city tour guide and amateur San Francisco historian, told me. “I’m sure that when the California Gold Rush happened all the people that were here from the beginning were like, ‘Look at all these people coming in and ruining our city. We can no longer find room to stay… prices are going up…’ They’ve been saying these things since 1849.”

Amster has a point. In 1848, when James W Marshall discovered gold along what’s now Highway 49 (an approximately 210km drive north-east of San Francisco), the then-tiny city’s population exploded. Tens of thousands of people flocked to the greater San Francisco Bay Area from across the US and abroad in that first year alone, all in search of fortune. Many of them were risk-takers who left their old lives for the promise of something better, and the chance to reinvent themselves in entirely new ways.

San Francisco became a boomtown practically overnight – a lawless place where gambling halls and brothels ran rampant and the male population far outweighed the female. This rugged and rebellious nature attracted a number of odd ducks and risk-takers: those like writer and humourist Mark Twain; Irish-born “Spanish” dancer Lola Montez; and Italian immigrant Domenico Ghirardelli, a pioneering businessman who founded the city’s Ghirardelli Chocolate Company. And its “anything-goes” attitude has remained a calling card for the city throughout its 170-plus-year history.

Emperor Norton was larger than life, as well as a true eccentric

From the literary-minded bohemians who infiltrated the city’s jazz clubs and coffeehouses in the 1950s to the Haight-Ashbury counterculture of the 1960s that helped birth the hippie movement, and the gay rights movement of the 1970s, a strong sense of liberal-mindedness and progressive thinking has been woven into San Francisco’s present-day fabric. But there’s one figure in San Francisco’s history that perhaps personifies the city and its ethos of tolerance better than anyone: Joshua Abraham Norton, the self-declared “Norton I, Emperor of the United States”, and later, “Protector of Mexico”.

“Emperor Norton was larger than life, as well as a true eccentric,” Amster said. He should know; he’s been leading his Emperor Norton’s Fantastic San Francisco Time Machine tours along the streets of the city’s downtown – and more recently its Embarcadero waterfront – for nearly a decade. Dressed much as the “emperor” often did on his jaunts around town, Amster brings guests on a roughly three-hour walking journey through both San Francisco history and the life of Norton himself. It’s undoubtedly an interesting one.

British-born Joshua Abraham Norton first arrived in San Francisco in the 1850s, and quickly made a comfortable living for himself in commodities markets and real estate. However, he soon lost it all with an ill-fated investment in Peruvian rice. Bankrupt and broke, Norton disappeared for a couple of years, with no-one knowing his whereabouts. Then on 17 September 1859, he reappeared (albeit a somewhat crazier version of himself), walking into the offices of San Francisco’s Daily Evening Bulletin newspaper and handing editor George Fitch a proclamation declaring himself, “Emperor of these United States”, at the “request and desire,” he said, “of a large majority of these United States”. Fitch printed the proclamation in that evening’s paper, and soon enough a new San Francisco legend was born.

I don’t know any other city that’s like this

“For 21 years, I ‘ruled’ the streets of San Francisco,” Amster tells his guests, performing in character throughout each tour. “Restaurants always gave me the best seats, I rode public transit for free and people would stand and bow whenever I entered the theatre, which I attended regularly.” Amster then shares the tale of how the emperor printed his own treasury bonds, backed by no guarantee, which bars and dining establishments citywide regularly accepted as legal tender; and explains that it was Norton himself who first declared anyone “uttering the abominable word ‘Frisco’” as an abbreviation for San Francisco would be faced with a £20 fine.

Over the course of the walk, tour participants start to understand just how ingenious the emperor was, and how beloved he soon became. They learn that he was a constant figure in the daily life of the city: appearing at public hearings to argue for equality; engaging local residents; and continuously writing proclamations that were quite progressive at the time, including standing up for African-Americans and their right to ride streetcars, and fighting anti-Chinese rhetoric.

He was very fair, said Amster, telling us that Jewish-born Norton attended the city’s still-in-operation Congregation Emanu-El every Saturday. “But he’d then go to church every Sunday,” Amster added, laughing. “He didn’t want one religion to get jealous of the other.” With his impartial attitude, which also included his defence of Native Americans and women and a personality that could easily be called idiosyncratic and unconventional, Emperor Norton represented what San Francisco has always been about. “That a man like him could come here,” said Amster, “and reinvent himself, and everyone says, ‘OK. If that’s who you are, then that’s who you are’ – I don’t know any other city that’s like this.”

These qualities of tolerance and acceptance – along with a penchant of encouraging its citizens to walk their own paths – are cemented in both the city’s boomtown days and in legendary figures like Emperor Norton, and they remain some of San Francisco’s most extraordinary features, despite the city’s ever-shifting demographics and public policies.

They’re visible in the works of local organisations like the SF Cacophony Society, a random group of renegades responsible for such events as the annual Valentine’s Day Great Pillow Fight and the original Burning Man (the ultimate freethinkers festival); in storytelling communities such as Muni Diaries, which share both live and online tales of random happenings – from acts of kindness to activism – that take place on San Francisco’s notorious public transit system and draw hundreds of submissions; and in treasured San Francisco personalities like Peaches Christ, a local drag queen whose cult-movie parody events such as Femlins (Gremlins) and Drag Becomes Her(Death Becomes Her) are some of the hottest tickets in town.

As for Emperor Norton, how would this man who so embodied the city’s progressive characteristics view San Francisco – with its massive Salesforce Tower dominating the city skyline, and buses for tech giants like Google lining up at stops previously used only for public transit – as it is now?

There are a lot of us working overtime to keep these traditions alive

“I think he’d feel mixed,” said Amster. “Deep down, remember, the emperor was a businessman. He’d probably appreciate the city’s niche bars and theatre scene, along with its financial successes, but more than anything he wanted things to be fair and to operate smoothly. I do believe he’d be very outraged at how those less fortunate are treated in the city nowadays, but knowing the emperor, he’d probably come up with a solution for it.”

Thankfully, there are many San Francisco residents and organisations working to address such issues, as well as continuing to find new ways to come together peacefully and open-mindedly. This might mean a tech “bro” and a sanitation worker sitting side-by-side sipping overpriced lattes in a cafe that bans laptop use on weekends to encourage communication; or them lining up next to each other for rotisserie chicken at one of the city’s gourmet fast-casual eateries, a compromise of high-end cuisine and (relatively) affordable prices that’s also a response to the city’s rising costs.

“Change in the city is inevitable,” said Amster, “but it can also be good, as long as we don’t erase those things that we hold near and dear about being San Francisco: the lack of conformity; the diversity; the spirit of reinvention. There are a lot of us working overtime to keep these traditions alive.”

Why We Are What We Are is a BBC Travel series examining the characteristics of a country and investigating whether they are true.

Join more than three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called “The Essential List”. A handpicked selection of stories from BBC Future, Culture, Worklife and Travel, delivered to your inbox every Friday.