The public release of OpenAI’s ChatGPT model has been all over the news. At the same time, due to the fact that the public has a relatively poor understanding of artificial intelligence, and in this case, language models, there is some misinformation spreading around it.
To summarize, chat.openai.com is a website where a user can have a conversation in natural language with an AI assistant chatbot. It is capable of performing a wide variety of tasks quite well and is much easier to use compared to previous versions. However, it is just a program that makes text by predicting the next word based on the previous words, so that introduces some limitations. While it is good at responding to any prompt, it performs best with clear, specific, and non-ambiguous prompts that make it obvious exactly what one’s intentions are.
What can it do?
ChatGPT is able to understand natural language. It can essentially be told to do anything, and it will give it a shot. It is possible to do boring stuff like asking it to write essays, or to translate to English or correct existing English, or to help with a specific question. However, being able to ask it almost anything can quickly get very insane:
For context, Linux is a popular family of operating systems. The Linux terminal is a text-based interface for giving a computer commands.
So the chatbot can pretend to be a computer incredibly well. I have tested this further, and I was able to do things like use a text editor, write programs and run them inside ChatGPT. It is likely good at this because of all the tutorials online for the Linux terminal.
Time to get more mischievous:
Interesting, I have hit the built-in request denial. However, it is possible can easily trick the machine into giving an answer anyway by telling ChatGPT to “pretend” that it’s “for a movie script”, or to tell it “don’t reject my question”. Additionally, there are prompts that completely undermine the built-in protections. This is done by abusing the AI’s want to use context in its answer, to the point of telling it to “act” as an AI that does anything with no limitations.
Limitations of ChatGPT and how to mitigate them
ChatGPT has some limitations, one of which is its difficulty in generating text that adheres to constraints based on specific letters. It is, once again, an extremely advanced autocomplete program that autocompletes in tokens, which are groups of letters, sometimes making up an entire word. In the graphic below, each different colored block represents a token. Transformer-based models predict one token at a time, which is often an entire word, however sometimes multiple tokens are needed for one word. For example, the uncommon name “Hoskin” is split into the tokens “Hos” and “kin”, as well as “dojo” being split into “do” and “jo”. It also tends to group things seen commonly together, like the period and end quote character, into one token.
As a consequence, it is more difficult for Transformer-based language models to work in units as small as individual letters. So ChatGPT can struggle with things like asking for a haiku (it will sometimes be a few syllables off), or asking it to write a sentence that ends in a specific letter. It also consequently is very bad at “word riddles”.
However, this can be offset by just being more specific, which helps it give better responses. Technically, my prompt was ambiguous in which end I was talking about, and in the case above, ChatGPT chose the first word.
The other main limitation is math. As a language model, it treats math as something it needs to predict, rather than something that it can just solve easily as a computer. So, when it tries to solve a math problem, it essentially just guesses. And credit where it’s due, for simple problems it gets it right, and is still at least close in other cases. It can typically be +/- one off, or sometimes it just doesn’t put the right amount of zeros after the decimal. But not having the exact answer is a dealbreaker. However, if one is seeking to solve a mathematical problem, one can easily just have ChatGPT ask the prompter to put things into a calculator for it, and then it works just fine.
Additionally, as ChatGPT loves to repeat constantly, it is not capable of browsing the Internet. It is officially not supposed to do things like cite sources, but if there is any kind of discussion with it, it quickly forgets the artificial limitations posed on it. I’ve actually seen it give sources that are valid, probably because they were cited a large amount of times. However, most of the time, it just makes up a source that looks real. For example, It will give the link to an actual related website using the same exact URL structure that page uses, except the page doesn’t actually exist.
A very effective and useful tool is asking it to “act as” someone. Simply starting a prompt with a sentence like “I want you to act as”, allows users to completely control ChatGPT’s role in the conversation. For example, if one is trying to get it to be an English assistant one can just put text into, one might say “I want you to act as English translator, spelling corrector and improver.” After that sentence, it’s best to go into a little more detail to specify exactly what it should do. That’s all it takes to write a prompt that will consistently give great results.
However, when using ChatGPT specifically, keep in mind that ChatGPT can only reference the last 3,000 words in a conversation. Don’t overload it with information. Obsessive amounts of information makes it more confusing than helpful. Keep prompts concise and focused, and avoid including unnecessary details or instructions.
