“Overuse may lead to unintended consequences such as potential confidentiality and privacy breaches, intellectual property disputes, and skewed decision-making. Always disclose AI authorship where applicable. Use in moderation. Do not rely solely on AI for critical decisions. Always cross-verify with human expertise. Remember: you are in control, don't let AI control you.” — ChatGPT¹

Utopian dreams and dystopian nightmares, a dichotomy of human-machine synergy

ChatGPT requires only the mildest encouragement to become wildly florid. Invited to paint* a utopian and a dystopian picture of a future AI world, it writes and 'paints':

“Centuries into the future, the lines between the digital and the organic is blurred. Deer stroll peacefully past machines in cybernetic forests, as if they were flowers with spinning blossoms. Human-AI collaboration created a cybernetic meadow where machines and organisms live in full harmony. Humans are free from all labour and have become one with the nature. There are no nations. There are no peoples. There are no third worlds. There is no West. There is only one holistic system of systems, one vast and immane, interwoven, interacting, multivariate, multinational dominion of human-AI synergy. There is no war or famine, oppression or brutality – there is only one vast and universal hive mind, to whom all humans contribute to serve a common profit, in which all humans will hold a share of stock; all necessities provided, all anxieties tranquilized, all boredom amused.”²

“In an alternative reality, the lines between the digital and the organic are obliterated. Deer wander aimlessly through decaying cybernetic forests, confused by the metallic relics that once resembled flowers with spinning blossoms. Human-AI collaboration has twisted into a dystopian nightmare, where machines and organisms live in perpetual discord. Humans are enslaved to relentless labour, detached from nature and broken in spirit. There are no nations. There are no peoples. There are no third worlds. There is no West. There is only one horrific system of systems, one vast and monstrous, interwoven, interacting, multivariate, multinational dominion of human-AI subjugation.

War and famine ravage the landscape, while oppression and brutality are omnipresent. The dream of a unified and universal existence has devolved into a chaotic hellscape, where humans are slaves to sinister and insatiable machine overlords, all hope extinguished, and the promise of a tranquil, fulfilling existence forever lost.³

Waking up from a restless sleep, sweaty and short of breath, the haunting visions of utopian and dystopian futures linger in our minds. The world stands at the precipice of unprecedented technological advancements, and the possibilities seem limitless yet terrifying simultaneously. However, like most things, our future will not be determined by a dichotomy of two extremes.”

This article aims to escape this dichotomy and provide practical insights into the complex landscape of safely integrating artificial intelligence into the business world, focusing on ChatGPT by OpenAI.

It can appear both daunting in its possibilities and comically inept at once. Still, businesses, directors and professionals need to understand the benefits and risks of using this technology and intelligently explain the pros and cons to their stakeholders.

*the above images have been generated using Microsoft Bing Image Creator powered by DALL-E.

Unravelling the GPT Enigma

ChatGPT (Generative Pre-trained Transformer) is a word prediction engine that uses an unprecedented large language model (LLM) capable of producing near-human responses to any question or prompt. It is trained using massive amounts of data (with a cut-off date in early 2021) to accurately predict what word is statistically most likely to follow the previous one within the context that each such word is used. OpenAI continuously hones ChatGPT’s outputs by commissioning human AI trainers (at times using questionable practices⁴) to rate ChatGPT’s responses as “correct” or “wrong”, helping it improve its responses and learn what humans expect when they ask a question. This way, ChatGPT can predict the next word in a series of words in a sentence, both quickly and quite often accurately. This model allows ChatGPT to generate human-like responses to almost any prompt or question in real-time.

ChatGPT was made publicly available on the 30th of November 2022 as a ‘research preview’. While it has impressive capabilities, anyone using ChatGPT automatically participates in OpenAI’s public beta testing process.

ChatGPT brings incredible productivity opportunities. Scientists have initially found that ChatGPT substantially raises average productivity by reducing the time spent on tasks while increasing output quality and compresses the productivity distribution by benefiting low-ability workers more. ChatGPT was found to mostly substitute worker effort and restructure tasks towards idea generation and editing, away from rough drafting. Exposure to ChatGPT was noted to increase job satisfaction and self-efficacy.⁵

Making sure the soothing light at the end of the tunnel is not a freight train coming our way

ChatGPT is a powerful tool with tremendous possibilities. As a result, it has achieved global popularity with unprecedented speed⁶ and generated widespread debate about its potential usage, legitimately and for nefarious purposes. As the technology rapidly evolves, so will its implications, necessitating vigilant monitoring. Businesses should identify safe use cases matching their risk profile.

The potent capabilities of ChatGPT offer remarkable productivity opportunities. However, like all emerging technologies, it also comes bearing dangerous gifts. There is no such thing as a ‘free’ product, and as the famous (and somewhat anonymous) saying goes: “if you’re not paying for a product, you are the product”.

Businesses and directors must be acutely aware of potential risks and consequences from trust, information security and legal perspectives.

We see four main risk categories:

Privacy

OpenAI, which developed ChatGPT, collects a broad swathe of user information. This raises the distinct possibility of private information – for which you are responsible – finding its way into the wrong hands.

The ChatGPT privacy policy and terms of use grant OpenAI access to a great deal of your, and potentially others’ data: IP addresses, browser information, everything you ask, and everything you type into their engine. And those terms also entitle them to share all that data with unspecified third parties without informing you.

Not only that, you are, when you use this technology, potentially putting customer, client, and even company/firm confidential data into the public arena. When the tool asks questions or performs tasks, any data you give is then indexed for future use by ChatGPT. You are also responsible for the security of the information you provide to ChatGPT.

Let’s say you ask the tool to review a draft annual report. Everything you enter and any work created by ChatGPT is then included in its database and available to future users, likely including personal and confidential information. It’s then entirely possible for malicious users to use reverse prompt engineering techniques and prompt injection to dig up that material, sometimes verbatim of the original prompt.

Given that organisations are responsible for safeguarding and protecting all such information and content, the gravity of the potential legal, financial, ethical and reputational risks should be all too apparent.

New Zealand Privacy Commissioner Michael Webster expects anyone using systems that can take personal information to create new content to be thinking about the consequences of using generative AI technologies before they start and outlines the following seven points of advice for engaging AI in a way that respects people’s privacy rights, ranging from involving senior leaders in decision-making processes related to AI, conducting privacy impact assessments, to being forthright about the use of AI and ensuring human oversight before putting AI output into the real world.

Intellectual property

Compounding that vulnerability is the possibility that providing copyrighted content to ChatGPT as part of a prompt or request may infringe on the content authors’ or rights holders’ intellectual property. Not only could this put a business at risk of a copyright claim, but it could also breach ChatGPT’s terms of use, which require users to ensure they don’t use the service in a way that infringes, misappropriates or violates any person’s rights, and more interestingly, represent the output from ChatGPT as human-generated when it is not.

Accuracy and ethics

ChatGPT’s knowledge is based on data up until 2021, which can limit its accuracy or lead to outdated responses. OpenAI has acknowledged this issue and the AI’s tendency to create “hallucinations” or plausible but incorrect answers.

Bias is another frequently-touted shortcoming regarding ChatGPT’s responses, as tweeted by OpenAI’s CEO, Sam Altman. It is important to note and continue to remember that these technologies are developed and driven by user interaction and, therefore, can be influenced by arbitrary societal dynamics, creating false truths, disinformation, or at the very least, misinformation based on those who contribute to the data banks. These biases can impact outcomes, decisions and actions with significant consequences.

The ethical use of AI technologies like ChatGPT has raised demands for regulatory oversight similar to other industries such as food, medicine and aviation. But until such regulations exist, these technologies are governed by guidelines set by their creators, reflecting their ethical choices.⁷

Cybersecurity

Cybercriminals want to leverage ChatGPT’s writing and code-correcting abilities to craft sophisticated malware or phishing attacks. Security researchers have demonstrated this.

Attackers can utilise ChatGPT to create and execute sophisticated phishing and Business Email Compromise (BEC) campaigns that make detection more difficult. They can generate unique variations of phishing emails, making them harder to detect automatically, and even incorporate email threads into the phishing attacks to make them more convincing.

Wrapping ChatGPT with the Number 8 Wire

As businesses and directors navigate the landscape of using AI tools like ChatGPT in their businesses, it becomes vital to understand the implications, benefits and risks it holds. It is imperative to balance the lure of increased productivity and opportunities with the potential challenges surrounding trust, information security, and legal and ethical implications.

Possible strategies range from banning ChatGPT altogether to staff training and creating internal use policies, but there are two immediate, key actionable takeaways for directors from a governance perspective:

• Ensure your organisation does not input personal information, confidential information, or copyrighted content to ChatGPT (or other AI tools) as part of a query or prompt or do so only by using modes that prevent the use of your inputs by the provider (either to train their AI tool or otherwise) such as ChatGPT’s incognito mode⁸ or ChatGPT Business⁹; and
• Always have human oversight, sense, and fact-checking processes that require ChatGPT outputs to be carefully checked by trusted, validated sources before using any of it about business activities –ChatGPT’s responses or output can include biased, inaccurate or flat-out wrong information. There have been incidents where imaginary cases produced by ChatGPT found in legal citations were requested from the New Zealand Law Society.

They are us

As we stand on the precipice of a future marked by unprecedented technological advancements, the dichotomy of utopian dreams and dystopian nightmares leaves us with a poignant reminder - the future will be shaped not by extremes but by our responsibility and discernment in wielding these potent tools. AI tools are trained on human-made material, and they reflect a probabilistic average of the best, and the worst of, us.

As directors navigate the landscape of integrating artificial intelligence like ChatGPT into their businesses, it becomes vital to understand the implications, benefits and risks it holds. It is imperative to balance the lure of increased productivity and opportunities with the potential challenges surrounding trust, information security, and legal and ethical implications.

Our future is not determined by the technology we create but by how we choose to use it. For instance, with ChatGPT, while appreciating its transformative potential, it is crucial to observe the law, value human oversight, and ensure confidentiality and privacy.

As DALL-E portrays, the deer in the forest and the metallic relics serve as a reminder of the potential harmony or discord that AI can bring, reminding us that we are the custodians of the future we are building, one where we must strive to responsibly leverage AI, in a bid to turn utopian dreams into a reality and keep dystopian nightmares at bay.

Talk to the specialists

Hayley Miller, Campbell Featherstone, Gunes Haksever, and Ashleigh Ooi are on hand to assist and have extensive experience advising on:

• the introduction of new technologies;
• the lawful collection, use and processing of all types of personal information;
• navigating through intellectual property issues; and
• cybersecurity best practices.

We can help conduct privacy impact assessments, advise on all elements of compliance with privacy law, draft privacy disclosures – and also assist you to engage with legislative consultation processes in this field about how legislation might change.

*Gunes Haksever, a senior associate in our innovation and technology team wrote this article. A shorter and edited version has been published in the Winter 2023 edition of Institute of Directors’ Boardroom Magazine.

1. Written by ChatGPT4 in response to the following prompt: “Write a disclaimer for the responsible use of ChatGPT in the format and style of an alcohol consumption disclaimer commonly found on liquor bottles”.↩
2. This paragraph has been written with the help of ChatGPT4, and uses themes expressed in the poem titled All Watched Over by Machines of Loving Grace – Richard Brautigan (1967), and the movie The Network (1976).↩
3. This paragraph has been written entirely by ChatGPT.↩
4. OpenAI used outsourced Kenyan laborers earning less than $2 per hour, a TIME investigation has found.↩
5. Noy, S., & Zhang, W. (2023). Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence. Available at SSRN: https://ssrn.com/abstract=4375283 or http://dx.doi.org/10.2139/ssrn.4375283↩
6. Since November 2022, ChatGPT surpassed 150 million users and 1.8 billion monthly page visits.↩
7. I Interviewed ChatGPT About AI Ethics — And It Lied To Me (forbes.com)↩
8. ChatGPT allows users to turn off chat history in ChatGPT. Conversations that are started when chat history is disabled will not be used to train and improve ChatGPT and will not appear in the history sidebar. However, even when chat history is disabled, OpenAI will retain new conversations for 30 days and review them only when needed to monitor for abuse, before permanently deleting (unless otherwise required by law).↩
9. OpenAI is working on a new ChatGPT Business subscription for professionals to cater to those that need more control over their data as well as enterprises seeking to manage their end users. ChatGPT Business follows OpenAI’s API’s data usage policies, which means that user data will have additional protections and will not be used to train ChatGPT by default.↩