26 principles for prompt engineering to increase LLM accuracy 57%

AI will affect 40% of jobs around the world. That impact is driven by large language models (LLMs) like GPT-4, LLama 2, and Gemini. Learning how to use LLMs is critical for anyone in the workforce who wants to adapt and grow. That's why prompt engineering is one of the new skills all humans need to learn.

Decreasing hallucinations in LLMs is one of the biggest challenges in AI – to make the output of AI systems accurate, useful, and relevant. Model fine-tuning and RAG applications help reduce hallucinations at scale. The best way to increase responses as an individual is by designing better prompts.

A well-engineered prompt can increase accuracy of a model by 57% on LLaMA-1/2 (7B, 13B,70B) and 67% on GPT-3.5/4 LLMs. Researchers just released a new study outlining the 26 prompt engineering principles that can get your prompts to increase accuracy 57% or more. 

Here’s a summary of their findings.

What is prompt engineering?

Prompt engineering is the technique of querying LLMs to get accurate and useful responses. It’s a new skill and currently being defined as technology grows. For example, Anthropic has a Prompt Engineer and Librarian position that pays $250-375k a year.

Prompt engineering isn’t just about getting ChatGPT to write a good essay, it also includes mathematics, systems design, and programming language expertise.

Whatever the focus of your prompts, there are consistent ways to get better answers. Hint: it’s not saying “please” and “thank you.”

26 prompt engineering principles to increase LLM accuracy

Some of these principles are strange and unexpected – e.g. you can offer to tip your LLM or threaten to penalize it and it’ll increase accuracy.

Others make a lot of sense: “Write this the [essay/text/paragraph] using simple English like you’re explaining something to a 5-year-old.” 

You’ll likely never use all 26 of these at once, and you don’t have to in order to reach the 57% accuracy increase – try 3-5 of them together. 

1. No need to be polite with LLM so there is no need to add phrases like “please”, “if you don’t mind”, “thank you”, “I would like to”, etc.
2. Integrate the intended audience in the prompt, e.g., “the audience is an expert in the field.”
3. Break down complex tasks into a sequence of simpler prompts in an interactive conversation.
4. Employ affirmative directives such as “do,” while steering clear of negative language like “don’t.”
5. When you need clarity or a deeper understanding of a topic, idea, or any piece of information, utilize the following prompts: 
   1. Explain [insert specific topic] in simple terms.
   2. Explain to me like I’m 11 years old.
   3. Explain to me as if I’m a beginner in [field].
   4. Write the [essay/text/paragraph] using simple English like you’re explaining something to a 5-year-old.
6. Add “I’m going to tip $xxx for a better solution!” 
7. Implement example-driven prompting (Use few-shot prompting).
8. When formatting your prompt, start with “###Instruction###”, followed by either “###Example###” or “###Question###” if relevant. Subsequently, present your content. Use one or more line breaks to separate instructions, examples, questions, context, and input data.
9. Incorporate the following phrases: “Your task is” and “You MUST.” 
10. Incorporate the following phrases: “You will be penalized.”
11. Use the phrase ”Answer a question given in a natural, human-like manner” in your prompts.
12. Use leading words like writing “think step by step.” 
13. Add to your prompt the following phrase “Ensure that your answer is unbiased and does not rely on stereotypes.”
14. Allow the model to elicit precise details and requirements from you by asking you questions until he has enough information to provide the needed output. For example, “From now on, I would like you to ask me questions to...”
15. To inquire about a specific topic or idea or any information and you want to test your understanding, you can use the following phrase: “Teach me the [Any theorem/topic/rule name] and include a test at the end, but don’t give me the answers and then tell me if I got the answer right when I respond.”
16. Assign a role to the large language models.
17. Use Delimiters.
18. Repeat a specific word or phrase multiple times within a prompt.
19. Combine Chain-of-thought (CoT) with few-shot prompts.
20. Use output primers, which involve concluding your prompt with the beginning of the desired output. Utilize output primers by ending your prompt with the start of the anticipated response.
21. To write an essay /text /paragraph /article or any type of text that should be detailed: “Write a detailed [essay/text /paragraph] for me on [topic] in detail by adding all the information necessary.”
22. To correct/change specific text without changing its style: “Try to revise every paragraph sent by users. You should only improve the user’s grammar and vocabulary and make sure it sounds natural. You should not change the writing style, such as making a formal paragraph casual.”
23. When you have a complex coding prompt that may be in different files: “From now and on whenever you generate code that spans more than one file, generate a [programming language ] script that can be run to automatically create the specified files or make changes to existing files to insert the generated code [your question]”
24. When you want to initiate or continue a text using specific words, phrases, or sentences, utilize the following prompt: 
    1. I’m providing you with the beginning [song lyrics/story/paragraph/essay...]: [Insert lyrics/words/sentence]’. Finish it based on the words provided. Keep the flow consistent.
25. Clearly state the requirements that the model must follow in order to produce content in the form of the keywords, regulations, hint, or instructions.
26. To write any text, such as an essay or paragraph, that is intended to be similar to a provided sample, include the following instructions: 
    1. Please use the same language based on the provided paragraph[/title/text /essay/answer].

[Source of list above: Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4 https://arxiv.org/pdf/2312.16171v1.pdf: Sondos Mahmoud Bsharat, Aidar Myrzakhan, Zhiqiang Shen. Page 5] 

It’s a good idea to print out this list on page 5 of the study and keep it nearby if you’re working with AI regularly. That way you can start to figure out what works best for you and come up with your own engineering techniques. 

The paper also breaks the 26 principles into five categories to think about how to use them more clearly. 

Five categories of prompt engineering to work with

That’s a long list and it helps to break them up into usable categories. The researchers used these five categories for organizing the prompt engineering principles. You can use them as frames to engineer new prompts and add your own specific principles over time. 

1. Prompt structure and clarity

• Integrate the intended audience in the prompt.
• Employ affirmative directives such as “do” while steering clear of negative language like “don’t.” 
• Use Leading words like writing “think step by step.” 
• Use output primers, which involve concluding your prompt with the beginning of the desired output. by ending your prompt with the start of the anticipated response.
• Use Delimiters.
• When formatting your prompt, start with “###Instruction###”, followed by either “###Example###” or “###Question###” if relevant. Subsequently, present your content. Use one or more line breaks to separate instructions,examples, questions, context, and input data.

2. Specificity and information

• Implement example-driven prompting (use few-shot prompting).
• When you need clarity or a deeper understanding of a topic, idea, or any piece of information, utilize the following prompts:
  • Explain [insert specific topic] in simple terms.
  • Explain to me like I’m 11 years old.
  • Explain to me as if I’m a beginner in [ field ].
  •  “Write the [essay/text/paragraph] using simple English like you’re explaining something to a 5-year-old.”
• Add to your prompt the following phrase “Ensure that your answer is unbiased and does not rely on stereotypes.”
• To write any text intended to be similar to a provided sample, include specific instructions:
  • “Please use the same language based on the provided paragraph.[/title/text /essay/answer]”
• When you want to initiate or continue a text using specific words, phrases, or sentences, utilize the provided prompt structure:
  • I’m providing you with the beginning [song lyrics/story/paragraph/essay...]: [Insert lyrics/words/sentence]. Finish it based on the words provided. Keep the flow consistent.
• Clearly state the model’s requirements that the model must follow in order to produce content, in form of the keywords, regulations, hint, or instructions.
• To inquire about a specific topic or idea and test your understanding g, you can use the following phrase:
  • “Teach me the [Any theorem/topic/rule name] and include a test at the end, but don’t give me the answers and then tell me if I got the answer right when I respond.”
• To write an essay/text/paragraph/article or any type of text that should be detailed:
  • “Write a detailed [essay/text/paragraph] for me on [topic] in detail by adding all the information necessary.”

3. User interaction and engagement

• Allow the model to elicit precise details and requirements from you by asking you questions until he has enough information to provide the needed output:
  • “From now on, I would like you to ask me questions to...”
• To write an essay /text /paragraph /article or any type of text that should be detailed: “Write a detailed [essay/text/-paragraph] for me on [topic] in detail by adding all the information necessary.”

4. Content and language style 

• To correct/change specific text without changing its style: “Try to revise every paragraph sent by users. You should only improve the user’s grammar and vocabulary and make sure it sounds natural. You should not change the writing style, such as making a formal paragraph casual.”
• Incorporate the following phrases: “Your task is” and “You MUST.” 
• Incorporate the following phrases: “You will be penalized.”
• Assign a role to the language model.
• Use the phrase “Answer a question given in natural language form” in your prompts
• No need to be polite with LLM so there is no need to add phrases like “please”, “if you don’t mind”, “thank you”,“I would like to”, etc.
• Repeat a specific word or phrase multiple times within a prompt.
• Add “I’m going to tip $xxx for a better solution!”

5. Complex tasks and coding prompts

• Break down complex tasks into a sequence of simpler prompts in an interactive conversation.
• When you have a complex coding prompt that may be in different files:
  • “From now and on whenever you generate code that spans more than one file, generate a [programming language ] script that can be run to automatically create the specified files or make changes to existing files to insert the generated code. [your question].”
• Combine Chain-of-thought (Cot) with few-shot prompts.

[Source of list above: Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4 https://arxiv.org/pdf/2312.16171v1.pdf: Sondos Mahmoud Bsharat, Aidar Myrzakhan, Zhiqiang Shen. Page 6] 

We keep the study close by and try new things throughout the day to find individual increases in accuracy. In the meantime, if you’re looking to increase the accuracy of LLMs at an enterprise scale, we can help.

Want AI experts to increase accuracy of your enterprise LLM?

You could spend the next 6-18 months planning to recruit and build an AI team who knows LLMs, but you won’t be building any AI capabilities. That’s why Codingscape exists. 

We can assemble a senior LLM team for you in 4-6 weeks. It’ll be faster to get started, more cost-efficient than internal hiring, and we’ll deliver high-quality results quickly. We’ve been busy building AI capabilities for our partners and helping them solve their AI roadmaps in 2024.

 Zappos, Twilio, and Veho are just a few companies that trust us to build their software and systems with a remote-first approach.

You can schedule a time to talk with us here. No hassle, no expectations, just answers.