Notes from ChatGLM 大模型应用构建 & Prompt 工程. Last Note is Here #4 #6.

Basic Knowledge

To-C vs. To-B

Pain Points of To-B

1. How to make LLM embedded in the process?
2. How to break down the workflow into actions?

Model & System

1. Manage user’s input so that model can understand better.
2. Manage model’s output so that it can interact with system well.
3. Make model focus on one specific kind of issues.
4. Make system control model for better questions.
5. Multiple model to co-work with each other.
6. Split long context to small pieces then summary (kind of like Map-Reduce).

Some Methods

1. Use flowchart to get a global view of whole process.
2. Evaluate the result then enhance.
3. Better test data makes better evaluation.

Prompt Engineering

Background

1. Lack of systematization and more relies on personal experience.
2. Hard to modify a prompt shared by others.
3. Depends on input data and needs evaluation tools.
4. Not reusable across models.

Techniques

Zero Shot

Few Shot

Chain of Thought

Zero Shot + CoT

Parameters

Temperature

Control the randomness of the output. Higher temperature gives more diverse answers.

Top-P

Kind of like Temperature.

Tricks

Splitter

1. Avoid other contents disturb the instruction for model.
2. Separate different contexts and modules.
3. Avoid irrelevant instructions.

Sub-Entry

1. Help model think step by step following the instructions.
2. Entry-by-entry writing & testing and iterative maintenance.

Knowledge Injection

Improve model’s accuracy and timeliness.

Formatted Output

1. May avoid overly diffuse and fabricated contents.
2. Easy to interact with other tools.

Avoid Peculiar Logic

Peculiar logic may confuse model.

Avoid Regularized Conversions

Leave it for regular program.

Nothing means None

Don’t force model to output.