In 2012, Richard Hawley was in Germany backstage with his band after the final night of a tour with Nancy Sinatra when the theatre producer Rupert Lord came to see him with an offer. “He said, ‘Your songs would make a wonderful musical.’” Hawley’s reaction? “Honestly, we just pissed ourselves laughing.”

But on a hungover ferry trip home the next day, something about the idea started to appeal: a musical set in Hawley’s hometown of Sheffield in the brutalist Park Hill housing estate overlooking the city. “It was such a curveball. And I thought, well, I know absolutely f*** all about musicals. I actually hate musicals. And I know nothing about theatre. So then I just thought – ‘F*** it. Let’s do it.’ Because it felt like the opposite of what I should be doing.”

As it turned out, Hawley’s music was a natural fit for the resulting award-winning production, Standing at the Sky’s Edge. He says his songs “come from the fabric of the city”, though by now they very much shape that fabric: a Sheffield native his whole life, the 57-year-old’s sumptuous, crooned, timeless songs are full of heart-on-sleeve romance, longing, hope, doubt, solitude and loss, often grounded in gritty realism. His solo albums have invariably been named after local landmarks (2005’s Mercury-nominated Coles Corner; 2007’s Lady’s Bridge, 2015’s Hollow Meadows).

Even the title of his vintage-sounding new album, In This City They Call You Love, pays tribute to the people of Sheffield, inspired by an overheard conversation from two out-of-towners sitting outside a café. “One of them just said, ‘Y’know, in this city, everybody calls each other love.’ I thought, that’s actually really beautiful. It’s a unique thing and we punctuate every sentence with it. There’s something disarming and charming, sincere and genuine about it.”

Standing At the Sky’s Edge, named after Hawley’s atypically angry Mercury-nominated 2012 album, has proved a revelation. Written by local playwright Chris Bush, it uses Hawley’s songs to flourish a time-hopping narrative that intertwines three generations of inhabitants of Park Hill: we see the working-class optimism when it is newly built in the 60s, the ruinous decline during the Thatcher-led decimation of the steelworks in the 80s, and its post-redevelopment revival in 2015. It is full of heart and deeply moving, touching on hot-button societal issues (capitalism, immigration, housing, gentrification).

Coming from a tribe of steelworking musician men – the women in his family all worked in the NHS – Hawley’s family saw all of this first-hand: in 1961 his grandparents were forced to move from Park Hill when the slums were cleared ahead of the regeneration; his dad and uncles were downcast by the miners’ defeat. “But I was adamant that it wasn’t going to be an exercise in political finger-wagging,” he says. “The story was enough on its own. More than anything, I wanted the voices of the residents of Park Hill that had been forgotten and abandoned to be heard.”

Still, in a post-industrial, post-austerity, cost-of-living landscape, it has found resonance beyond its locale. “It’s universal,” he says. “I think we’ve touched a nerve.”

It opened at the Sheffield Crucible in 2019 to rave reviews, since moving to London’s National Theatre, and it is currently in the West End at Gillian Lynne Theatre until August. It has won a host of awards: Hawley, together with orchestrator Tom Deering, won an Olivier for Best Original Score. When I bring up his bewildered, wayward, sweary acceptance speech – cut off on TV by the time he was paying tribute to Pulp bassist Steve Mackey, “my oldest friend”, who had died shortly before – he laughs his surprisingly shrill guffaw. “It was reflective of how we all felt, because we gatecrashed the West End. We don’t belong there.”

Hawley remains more working men’s club than West End, perhaps the only mainstream songwriter who could end up in the elite world of the latter while writing a song inspired by the former: rumbling new track “Two for His Heels” is a phrase taken from hearing some older gentlemen playing cards in his local. “Jock in the pub next door will be playing cribbage and you’ll always hear him say, ‘One for the Jacks and two for his heel!’ To me that’s surrealist poetry. I’ve taken it out of context. I love doing that.”

Hawley is on video call from home; he has the screen off after a particularly heavy night with his band and members of Arctic Monkeys. “It’s me and the dogs laid on the bed and you don’t want to see that, mate.” Nothing unusual there – Hawley loves a drink – but this was a poignant occasion toasting the life of Duane Eddy, the legendary American guitarist and Hawley’s influence, collaborator and friend, who died the day before at the age of 86. “It was a sad but lovely night. We ended up really close, a lovely, lovely man. I’m feeling a bit sad, but that’s normal. I go with how I feel.”

It was sadly appropriate: for In This City They Call You Love, his excellent, trademark set of richly textured, atmospheric pre-Beatles rock and balladry – “I’m not going to make a Finnish throat-singing hip-hop record” – is somewhat shadowed by death. “I’ve suffered quite a lot of loss of life.” In the songs demo-ed and in parts recorded organically from his garden shed Disgracelands, Hawley plays guitars gifted to him by Eddy (“too precious to take out the house”) and the family of the late Scott Walker, as well as instruments that belonged to his father, who died in 2007.

It’s there in the songs, too: the lushly orchestrated “Heavy Rain”, he says, “was inspired by how I felt after Steve left us. And there’s a lot of that (in the album). But to keep going, you’ve got to make peace with it. And I guess my healing process is putting it gently and with respect into the songs. I think that’s healthy. This record is me at peace.”

Before going solo in 2001, Hawley played with two bands, Treebound Story and Britpop outsiders Longpigs, and was saved from a spiralling drug addiction by becoming a touring member of Pulp in 1998. But he’d been performing with his dad and uncles since the age of 12 at weddings and in pubs; during his summer holidays at 14, he toured across Europe with his uncle Chuck. “It was really heavy. He lied to my mum and told her we were playing nice theatres. I just remember some pretty scary places with Russian sailors pissed out their minds on vodka and Polish sailors fighting in the streets. It was nuts.”

Had things been different, a life down the mines “would have been a distinct possibility. Growing up in Sheffield at that time, your choice was extremely limited.” But that was immaterial: instead, he watched as the dismantling of the mining industry during the strikes devastated the city, and his family. “It was savage. The hammer blow for Thatcher’s Britain fell hardest here, because she had to smash the unions. And I think people were just lost.”

He tells me a characteristic story about his stepfather, Stuart, a miner present at the Battle of Orgreave who lost his livelihood, and sense of purpose. “And he couldn’t forget about it. And he certainly couldn’t forgive. It was something that he took to his grave, that anger. It was a life that wasn’t lost, it was stolen. And for profit.”

He therefore finds parts of the musical hard to watch. “Because it’s so near the knuckle to what happened to my family. Poverty got so bad in the 80s. I knew great men who just fell apart in front of me. I’m admitting this to myself, which I guess I’ve never said, but it’s still raw. It doesn’t diminish with time. I wish it would. It just doesn’t.”

His political views are therefore deeply entrenched. “I’d rather shit in my own hands and clap than vote Tory. I think we’ve become a laughing stock around the world. We’re talking about men losing their pride – but I think we have as a nation.”

As a Labour voter, his hope is that they win power and “save the NHS” – “it’s a beacon in the dark and these f***ers have crippled it on purpose” – and turn the country into “something that works for everyone and not for six or seven multibillionaires. I’m not a communist, but I do believe that decisions need to be made for the benefit of the majority. That to me seems completely logical.”

Hawley says he’s tried to write political songs, but always comes back to the personal. “Those involved in the fallout from stupid politicians that affect what is often called ‘ordinary people’. But there’s nothing ordinary about people.” Community, friends, family; it’s the crux of Hawley’s world. He’s been with his wife Helen, a psychiatric nurse, for 33 years. “A miracle. She needs a medal as big as a bin lid.”

And yet loneliness is a constant theme of his work. New songs “Prism in Jeans” and “Hear That Lonesome Whistle Blow” are just the latest to draw from a mournful sense of isolation. “Well, happy songs generally are shit,” he says with a laugh. But he admits: “I have got this loneliness that I can’t hide. I can’t shake it either. And I don’t know why. But it’s there.” He says it might be the 10 years he spent on the dole from the age of 16, “a sort of lost youth”.

“But maybe it comes from seeing what happened to all those men,” he says. “It was like living through a war. A cultural one as well as physical. It sits not very comfortable in my psyche. But if I knew the specific answer, I wouldn’t need to write those songs. I’d probably be working in the chippy down the road. Viva loneliness!”

His voice quietens. “It’s an odd thing, isn’t it, that something that potentially could be such a negative has ended up being my fortune?”

‘In This City They Call You Love’ is out now.

‘Standing At the Sky’s Edge’ runs at Gillian Lynne Theatre in London until 3 August.

Richard Hawley tours the UK from 2 June (richardhawley.co.uk)