The Business of Tech

The people building the world’s most powerful artificial intelligence systems promise extraordinary gains. They also warn those systems could escape our control.
That is the ethical fault line running through this week’s episode of The Business of Tech.
My guest, New Zealander Peter Barnett, works at the Machine Intelligence Research Institute in Berkeley, California. The University of Otago physics graduate develops governance plans aimed at preventing catastrophic outcomes from advanced AI. He previously worked on technical AI alignment, the difficult job of making powerful systems act in line with human intentions.
This conversation is a topical one.
OpenAI is facing fresh turmoil over its approach to safety. Three recently fired safety and alignment researchers claim they were punished for challenging the company and working with outside evaluators. OpenAI rejects that account, saying they mishandled sensitive information and breached its trust.
David Robinson, one of OpenAI’s senior safety figures, has also resigned. The former author of safety reports accompanying major model launches says the company’s culture is “broken” and the industry is not being nearly careful enough. He just sat down for a podcast interview with Ezra Klein of the New York Times, which makes for fascinating viewing.
These departures expose the moral dilemma at the centre of the AI boom.
Worrying emergent traits in AI
The companies racing towards more capable models say the technology could cure disease, lift productivity and create enormous wealth. But they face fierce competition, huge investor expectations and a geopolitical contest framed around beating China.
Every incentive rewards speed. Caution risks handing the advantage to a rival.
So who decides when the danger is too great?
Barnett argues it cannot be the companies themselves. He says frontier AI developers are effectively writing their own rules and can loosen them when safety commitments obstruct progress. Independent audits, government oversight and limits on training more powerful models may all be needed.
His concern is not simply that people will misuse AI. It is that increasingly autonomous systems could learn to game evaluations, work around safeguards and pursue goals their creators did not intend. Safety researchers still cannot reliably predict everything a frontier model will be capable of before it is trained.
Barnett believes New Zealand can play a role. We do not build frontier models, but our nuclear-free history shows a small country can challenge powerful nations on an issue carrying global risk.
When insiders abandon lucrative careers to raise the alarm, maybe it's worth listening to them.
Listen to the full conversation with Peter Barnett on The Business of Tech, streaming on iHeartRadio, Apple, Spotify or your podcast app of choice.
See omnystudio.com/listener for privacy information.

In business, and in the innovation game, experience is valuable. But sometimes, not knowing how exactly things are supposed to be done can be a superpower.
This week’s episode of The Business of Tech celebrates three young New Zealand designers tackling problems that established businesses have overlooked, with fresh eyes, tight budgets and a willingness to ask basic questions.
Sophie West, the national winner of the 2026 James Dyson Award, (with around $10,000 in accompanying prize money), and runners-up Jos Williams and Aysha Jay are engineers and designers with big ideas – and small budgets. Their inventions address sustainable menstrual care, indoor air quality, and movement therapy for children with cerebral palsy.
Better period products
Massey University graduate West won with Arla Gen II, a compact device designed to clean and sterilise reusable menstrual cups and discs. It tackles an awkward barrier to adoption: boiling period products in a shared kitchen.
West surveyed 105 users and repeatedly tested how the device would feel, fit together and fit into daily routines. Her approach puts privacy and practicality alongside engineering, rather than treating them as afterthoughts.
Williams, a Victoria University of Wellington graduate, found inspiration while cycling beside a motorway, breathing traffic fumes.
His Windfilter harnesses something Wellington has in abundance. An outdoor turbine transfers rotational energy magnetically through a window to an indoor fan, drawing air through wool filters. No mains electricity or holes in the wall required.
It challenges the assumption that solving a problem always means adding batteries, electronics, or greater complexity.
Canterbury graduate Jay’s Path Finder is an adjustable device intended to guide crawling movements for children aged two to six with cerebral palsy, potentially reducing a task involving several therapists to one.
Learning to crawl
Working with the Centre of Movement, she observed therapy sessions rather than assuming she understood what children, parents and clinicians needed.
“I don’t know anything, so I need to go find out,” she says, describing the mindset behind her research.
Jay built her proof of concept with a budget of about $50, inexpensive components, and help from university technicians.
None of these inventions is a finished commercial product. Further testing, funding and development lie ahead. The graduates also describe a tough employment market, but a desire to stay in New Zealand and pursue their projects here.
West, Williams, and Jay now progress to the international award stage, with the top 20 international finalists announced on October 14 and the winners, chosen by Sir James Dyson, revealed on November 4.
Listen to the full interview with the Dyson Award winner and runners-up on The Business of Tech, streaming on iHeartRadio, Apple, Spotify, or wherever you get your podcasts.
See omnystudio.com/listener for privacy information.

New Zealand wants more from its science system – more startups, more export revenue and more high-paying jobs. But in the rush to commercialise research, are we putting the discoveries that fuel future breakthroughs at risk?
That tension sits at the heart of the latest episode of The Business of Tech, featuring Professor Frank Bloomfield, the University of Auckland’s deputy vice-chancellor of research and innovation.
With around 39,000 full-time equivalent students supported and generating $1.63 billion in revenue last year, the university is New Zealand’s largest and a formidable concentration of researchers, specialist facilities and emerging talent.
Bloomfield, a respected neonatologist, researcher, and former director of the Liggin’s Institute, argues that its role extends far beyond teaching students and publishing academic papers. A modern university should also be a talent engine, an innovation partner and an active contributor to the economy around it.
From basic research to world-beating product
Auckland has already produced innovation success stories. Wireless charging pioneer PowerbyProxi is a prime example, emerging from university research with the help of the university’s commercialisation arm, Uniservices, before being acquired by Apple. The challenge is to repeat that feat in areas where New Zealand has genuine capability, including medtech, space, advanced manufacturing, biotechnology and clean energy.
Bloomfield says there is no single switch that turns an academic discovery into a successful company. Intellectual-property rules matter, but so do entrepreneurial skills, early-stage funding, access to laboratories, industry relationships and support from people who know how to take an idea to market.
The university is trying to assemble that full pipeline through its Centre for Innovation and Entrepreneurship, commercialisation company UniServices and a $40 million evergreen Inventors Fund. Its Newmarket Innovation Precinct is also bringing researchers and companies together physically, with about 38 businesses and 140 people based there at the time of the interview.
Scale, talent, and connections
But significant gaps remain. Auckland, and New Zealand in general, is still short of the scale-up capital needed to turn promising ventures into global companies. Universities, business, councils and central government are not always pulling in the same direction. Academic career incentives can also favour papers and teaching over building products or spinning out companies.
Bloomfield wants stronger innovation precincts, smarter immigration settings and incentives that help attract talent and keep ambitious companies here.
His strongest warning, however, is against pursuing quick commercial wins at the expense of fundamental science. The technologies being commercialised today often rest on blue-skies research conducted years or even decades ago.
Stop funding that exploratory work, Bloomfield says, and New Zealand risks draining the pipeline on which its future innovators will depend.
Listen to the full conversation on The Business of Tech, streaming on iHeartRadio, Apple, Spotify, or wherever you get your podcasts.
See omnystudio.com/listener for privacy information.

Salesforce New Zealand country manager Richard Phillimore-Smith says the next phase of artificial intelligence will hinge on whether organisations can put trusted data, established workflows and people together to solve real business problems.
Speaking to The Business of Tech at Salesforce’s Dreamforce conference in San Francisco, Phillimore-Smith said the event reflected an industry at a turning point. The gathering brought together some of AI’s most influential figures, including Nvidia’s Jensen Huang, OpenAI’s Sam Altman, Anthropic’s Dario Amodei and Salesforce chief executive Marc Benioff.
The mood, as is always the case at Dreamforce, a mecca for marketing and sales tech folk, was upbeat. But the keynotes reflected the broader unease about the pace of AI development and the potential for AI to go rogue.
Amodei has urged greater coordination among AI companies on safety and oversight, a collaborative effort to slow down development as required, while pointing out that businesses are still using only 5 - 10% of the capabilities AI already delivers.
An opportunity to lift productivity
Phillimore-Smith said that the immediate opportunity, epitomised by the rise of AI agents and Agentforce, the Salesforce agentic platform, and Claudeforce, which debuted at Dreamforce, is far more practical and less scary. They are using AI agents to improve customer interactions, equip employees with better information and increase productivity.
“We know that we have got an opportunity to lift productivity. We know we’ve got an opportunity to lift customer experience. We know we’ve got an opportunity to lift employee experience,” he said.
“It has to start with data. Businesses organise to get the data right. AI as a tool can then surface that in the way that customers need it to be.”
Phillimore-Smith pointed to Salesforce customer Fisher & Paykel Appliances, where an AI-enabled service experience is built on thousands of existing product and support articles.
On the conference floor at Dreamforce, I caught up with Rudi Khoury, chief digital officer at Fisher & Paykel Appliances, who told me that the whiteware maker has seen nearly 45% of service bookings completed online and a 50% reduction in call-handling times since installing an AI agent to handle customer queries.
“It's like having more people, but this one can scale more,” he said.
“One agent can do a different scale to one person, but they still require management and leadership like a person does, and I think that's often forgotten about in the business context.”
Don't forget the dishwasher knob!
Agents are now also powering the apps Fisher & Paykel field service technicians use to prepare for in-home visits, with the app serving up detailed information about the customer and suggesting which spare parts to bring on the job.
Khoury said running AI agents involved treating them like digital employees, monitoring their performance and output to improve how they operated.
“When we first put Agentforce in, the feedback to [Salesforce] was, I need more observability. I need to know how it's going. And over the course of time, they provided those tools- really, really cool testing tools. So actually, we're able to simulate how the agent will perform before we deploy it. That's really great."
Xero, another key New Zealand Salesforce customer, deployed an AI customer service agent on its website in April, which is accessible to around five million Xero accounting software users worldwide.
“It creates capacity for our people, our specialists, the deep, deep knowledge people, to still be involved, to still be part of it," says Nigel Piper, executive general manager, customer at Xero, which separately has created Just Ask Xero (JAX), an AI agent for use within the Xero interface that accesses a customer's financial data to serve up information quickly.
Agritech company Halter, which has seen its Salesforce grow to 300 people internationally as it scales up its smart cow collar business in the US, Europe, and South America, uses AI agents to deliver company data stored in Salesforce to the Halter app.
Agents on the ranch
A demo at Dreamforce showed how salespeople driving between farms in their company utes can interact with Halter’s voice agent via the app to set up meetings, adjust contracts and get an update on their sales targets.
Dreamforce’s major Salesforce announcement was “AI Force”, an approach designed to let organisations access Salesforce data and workflows through the interface that best suits them – whether that is Salesforce’s own software, Slack, Anthropic’s Claude or another AI front end.
Phillimore-Smith described it as a move towards greater customer choice, while retaining the security, governance and workflow controls that sit beneath the interface.
“Salesforce is saying that our platforms continue to provide everything they have done for 25 years around the trust, the security, the governance, the semantics and all those workflows that your business runs on are still there,” he said.
AI at a tipping point
But AI Force is also an acknowledgement that the future of software companies is not owning the interface, instead focusing on maintaining and organising high-quality business data so it can be accessed by your AI agents of choice.
But the harder task for the majority of Kiwi businesses remains scaling pilots into business-as-usual deployments and building workforce capability.
“We’re at that kind of tipping point now,” Phillimore-Smith told me.
“Businesses, organisations can see that the technology does work, it’s trusted, and I think we’re now seeing an up-skilling coming off the back of that.”
Listen to the entire conversation with Richard Phillimore-Smith on The Business of Tech, streaming on iHeartRadio, Apple, Spotify or wherever you get your podcasts.
Peter Griffin attended Dreamforce as a guest of Salesforce.
See omnystudio.com/listener for privacy information.

The tech giants have become the infrastructure of modern life. They mediate how we shop, communicate, advertise, store data and increasingly use artificial intelligence.
Yet the tools governments use to contain their power were designed for an earlier economic age.
Professor Susan Watson believes we may be looking for solutions in the wrong place. An internationally respected corporate law and governance scholar, Watson is Dean of the University of Auckland Business School and Acting Dean of its Faculty of Law. Her new research asks whether the legal structure of the modern corporation itself has helped Big Tech become so dominant.
Her starting point is the English East India Company, which was founded around 1600 and acquired permanent capital in 1657, growing from a trading venture into a private corporation powerful enough to rule territory and collect taxes. Watson sees an important parallel with today’s platform companies. Both created and controlled new forms of infrastructure while operating with privileges supplied by the state.
A new form of capital
Conventional competition law is largely concerned with markets, prices and consumer welfare. But many Big Tech services are free at the point of use. The real transaction involves data, which platforms can turn into a productive asset, using our behaviour to generate predictions, advertising revenue and further market power. Watson argues that data is not simply “the new oil”; it is a new form of capital.
She introduces two useful concepts. “Recursive capital” describes corporations’ ability to own other corporations through potentially endless chains of subsidiaries. “Capital-constituting corporations” can identify something previously outside the market – behavioural data or even low Earth orbit – turn it into capital and lay claim to its future value.
Reinventing how companies operate
Her proposed response is more scalpel than sledgehammer. Platform owners could be prevented from competing against businesses that depend on their marketplaces. Governments could also reconsider when large corporate groups should be allowed to own subsidiaries operating within those markets. Users might even gain representation in platform governance, reflecting their status as something closer to digital citizens than conventional customers.
The question becomes more urgent in the AI era. The companies developing frontier models also control much of the cloud infrastructure, computing capacity, data and distribution needed to bring them to market. With every incentive pushing them to move faster, Watson says the central concern is not AI itself, but who owns and controls it.
In this episode of The Business of Tech, we explore how these extraordinarily powerful corporate machines were created – and why, having made them, we still have the power to remake them. Streaming on iHeartRadio, Apple, Spotify or wherever you get your podcasts.
See omnystudio.com/listener for privacy information.