思维链 Chain-of-Thought (COT) 变体篇¶
来自：AiGC面试宝典
宁静致远
2024年01月27日19:14
- 思维链 Chain-of-Thought (COT) 变体篇
- 思维链 Chain-of-Thought (COT)：思维链的启蒙
-
- 什么是思维链 Chain-of-Thought (COT)？
-
- 思维链 Chain-of-Thought (COT) 是思路是什么？
-
- 思维链 Chain-of-Thought (COT) 存在问题？
- 思维树 Tree of Thoughts (TOT)：一种用树结构解决复杂问题的方法
-
- 为什么需要思维树 Tree of Thoughts (TOT)？
-
- 什么是思维树 Tree of Thoughts (TOT)？
-
- 思维树 Tree of Thoughts (TOT) 涉及问题有哪些？
- 思维树 Tree of Thoughts (TOT) 解决复杂任务实例
- example 1: LLMs to Game of 24
- example 2: LLMs to Create Writing
- example 3: LLMs to 5x5个迷你填字游戏
- 思维图 Graph of Thoughts (GOT)：一种把思维链过程建模层图结构的方法
-
- 为什么需要思维图 Graph of Thoughts (GOT)？
-
- 什么是思维图 Graph of Thoughts (GOT)？
-
- 思维图 Graph of Thoughts (GOT) 核心思想是什么？
- 思维算法 Algorithm of Thoughts (AOT)：一种用DFS/BFS示例解决问题的方法
-
- 为什么需要思维算法 Algorithm of Thoughts (AOT)？
-
- 思维算法 Algorithm of Thoughts (AOT) 思路是什么？
-
- 思维算法 Algorithm of Thoughts (AOT) vs 其他 COT 的区别？
- 思维链 Chain-of-Thought (COT) 有哪些应用场景？
- Task 1: 复杂任务求解
- Task 2: 增强特定任务可靠性
- 思维链 Chain-of-Thought (COT) 有哪些局限性？
- 致谢
扫码查看更
思维链 Chain-of-Thought (COT)：思维链的启蒙¶
论文名称：Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
论文地址：
https://proceedings.neurips.cc//paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abc
a4-Abstract-Conference.html
1. 什么是思维链 Chain-of-Thought (COT)？¶
思维链（Chain-of-thought）提示使大型语言模型能够处理复杂的算术、常识和符号推理任务。强调了Chain-of-thought推理过程。
Standard Prompting¶
Model Input¶
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: The answer is 11.
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
Model Output¶
A: The answer is 27.
Chain-of-Thought Prompting¶
Model Input¶
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. $ 5 + 6 = 11 $. The answer is 11.
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
Model Output¶
A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have $ 3 + 6 = 9 $. The answer is 9.
2. 思维链 Chain-of-Thought (COT) 是思路是什么?¶
思维链 Chain-of-Thought（COT）类似于一个解决方案，它模仿了一个逐步思考的过程来得出答案（或者solutions/explanations通常在final answer之后）。
Math Word Problems (free response)¶
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.
Math Word Problems (multiple cho¶
StrategyQA¶
Q: How many keystrokes are needed to type the numbers from 1 to 500? Answer Choices: (a) 1156 (b) 1392 (c) 1480 (d) 1562 (e) 1788
Q: Yes or no: Would a pear sink in water?
A: There are 9 one-digit numbers from 1 to 9. There are 90 two-digit numbers from 10 to 99. There are 401 three-digit numbers from 100 to 500. $ 9 + 90(2) + 401(3) = 1392 $. The answer is (b).
A: The density of a pear is about 0.6 g/cm $ ^{3} $, which is less than water. Thus, a pear would float. So the answer is no.
SayCan (Instructing a robot)¶
Human: How would you bring me something that isn't a fruit?
Date Understanding¶
Q: The concert was scheduled to be on 06/01/1943, but was delayed by one day to today. What is the date 10 days ago in MM/DD/YYYY?
Explanation: the user wants something to eat that isn't a fruit. An energy bar is not a fruit, so I will bring the user an energy bar.
Plan: 1. find(energy bar) 2. pick(energy bar) 3. find(user) 4. put(energy bar) 5. done().
A: One day after 06/01/1943 is 06/02/1943, so today is 06/02/1943. 10 days before today is 05/23/1943. So the answer is 05/23/1943.
Last Letter Concatenation¶
Q: Take the last letters of the words in "Lady Gaga" and concatenate them.
A: The last letter of "Lady" is "y". The last letter of "Gaga" is "a".
Concatenating them is "ya". So the answer is ya.
CSQA (commonsense)¶
Q: Sammy wanted to go to where the people were. Where might he go?
Options: (a) race track (b) populated areas (c) desert (d) apartment (e) roadblock
A: The answer must be a place with a lot of people. Race tracks, desert, apartments, and roadblocks don't have a lot of people, but populated areas do. So the answer is (b).
Sports Understanding¶
Q: Is the following sentence plausible? "Joao Moutinho caught the screen pass in the NFC championship."
A: Joao Moutinho is a soccer player. The NFC championship is part of American football, not soccer. So the answer is no.
Coin Flip (state tracking)¶
Q: A coin is heads up. Maybelle flips the coin. Shalonda does not flip the coin. Is the coin still heads up?
A: The coin was flipped by Maybelle. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is no.
3. 思维链 Chain-of-Thought (COT) 存在问题?¶
思维链 Chain-of-Thought (COT) 需要 LLM的参数量级必须大
eg: 论文使用效果最好的是PaLM 540B的模型，看来模型越大对思维链的能力就越强
思维树 Tree of Thoughts (TOT)：一种用树结构解决复杂问题的方法¶
论文名称：Tree of Thoughts: Deliberate Problem Solving with Large Language Models
论文地址：https://arxiv.org/abs/2305.10601
论文 Github：https://github.com/princeton-nlp/tree-of-thought-llm
1. 为什么需要思维树 Tree of Thoughts (TOT)?¶
CoT通常只有一条解决问题的路径，但是有一些复杂问题的答案并不止有一条解决问题的路径，而是由多条解决问题的路径组成。
2. 什么是思维树 Tree of Thoughts (TOT)？¶
思维树 Tree of Thoughts (TOT) 把一条reasoning路径拓展至多条reasoning paths，这样模型可以综合多条reasoning path的结果得到最终的结论。
(a) Input-Output (c) Chain of Thought Prompting (IO) Prompting (CoT)
(c) Self Consistency with CoT (CoT-SC)
注：LLM解决问题的各种方法。每个矩形框代表一个thought，它是一个连贯的语言序列，是解决问题的中间步骤。ToT将任何问题定义为在树上的搜索，其中每个节点都是一个状态s=[x; z_1i]，表示到目前为止具有输入和thought序列的部分解决方案
3. 思维树 Tree of Thoughts (TOT) 涉及问题有哪些?¶
1. Thought Decomposition¶
a. 问题描述：如何将中间过程分解为思维步骤；
2. Thought Generator¶
a. 问题描述：如何从每种状态中产生潜在的thought；
b. 任务定义：给定一个树的状态，从k个候选项中决定下一个step
c. 实现方法：采用采样和投票的方法
3. State Evaluator¶
a. 问题描述：如何启发式地评估状态；
b. 任务定义：给定不同状态的边界，状态评估器评估他们在解决问题方面取得的进展，并作为搜索算法的启发式算法，以确定要继续探索哪些状态以及以何种顺序进行探索
c. 实现方法：
i. value类型的分类或者打分
ii. 投票的形式
4. Search algorithms¶
a. 问题描述：使用什么搜索算法。
b. 实现方法：可拔插式的使用不同的搜索算法
思维树 Tree of Thoughts (TOT) 解决复杂任务实例¶
example 1: LLMs to Game of 24
·任务介绍：Game of 24 游戏是数学推理游戏，目标是通过4个数，然后加减乘除得到24
· LLMs to Game of 24 思路：输入是“4 9 10 13”，最终得到的答案是“(10 - 4) * (13 - 9) = 24”，产生了3个intermediate thoughts可以是：“13 - 9 = 4 (left: 4 4 10); 10 - 4 = 6 (left: 4 6); 4 * 6 = 24 (left: 24)”，ToT会产生很多的这样的intermediate steps，然后我们采用BFS来执行这些steps。
example 2: LLMs to Create Writing¶
·任务介绍：给定四个句子，然后产生4个paragraph，分别以这4个句子结尾
· LLMs to Create Writing 思路：
·首先产生k=5个候选的plan，投票选择最佳的passage，基于最佳的plan，产生5个passage；
然后选择其中产生的最佳passage，投票的过程是一个zero-shot的过程
example 3: LLMs to 5x5个迷你填字游戏¶
·任务介绍：
·LLMs to 5x5个迷你填字游戏 思路：探索LM作为一个探索自己思想的一般问题解决者的极限并以深思熟虑的推理作为启发法来指导自己的探索。
思维图 Graph of Thoughts (GOT)：一种把思维链过程建模层图结构的方法¶
论文名称：Tree of Thoughts: Deliberate Problem Solving with Large Language Models
论文地址：https://arxiv.org/abs/2308.09687
论文 Github：https://github.com/spcl/graph-of-thoughts
1. 为什么需要思维图 Graph of Thoughts (GOT)？¶
· CoT通常只有一条解决问题的路径；
· TOT 解决问题的路径类似一颗树；
但是有时候解决问题的父级节点不只有一个，而是类似拓扑图的形式。
2. 什么是思维图 Graph of Thoughts (GOT) ?¶
思维图 Graph of Thoughts (GOT) 通过构建了一个有向图来解决问题。
3. 思维图 Graph of Thoughts (GOT) 核心思想是什么？¶
GoT系统结构包含一系列的交互模块：
· Prompter：为LLM准备消息；
·Parser：从LLM的回复中提取信息；
· Scoring module：验证LLM回复并对其进行评分；
· Controller:
· 介绍：协调整个推理过程，并决定如何进行推理
· 重要元素：
• Graph of Operations (GoO)：一种静态结构，它指定了给定任务的图分解，它规定了应用于LLM思想的转换，以及它们的顺序和依赖关系
• Graph Reasoning State (GRS)：一个动态结构，它保持正在进行的LLM推理¶
过程的状态（其思想及其状态的历史）¶
Legend¶
API for Controller¶
//LLM params: model used, temperature, max tokens, api key, org, ...
//LLM cost features: prompt token cost, response token cost, ...
//Instances of Prompter + Parser + Graph of Operations,
//Any additional input parameters (e.g., numbers to be sorted).
Available operations when building GoO (extensible)¶
Generate, Aggregate, Score, ... //see Prompter API
KeepBest(N) //preserves N best scoring thoughts
Repeat(k) //Repeat a given operation k times, generating k thoughts.
//For example, this enables "Aggregate" to generate multiple outcomes
//of the combination operation. Each such thought is maintained
//within the Graph Reasoning State and scored individually.
API for Prompter (extensible)¶
= Generate(t,k) //generate a prompt for k new thoughts, using thought t
= ValidateAndImprove(t) //generate a prompt to enhance thought t,
= Aggregate(t1, ..., tk) //generate a prompt to combine thoughts t1, ..., tk
= Score(t) //score thought t
= Validate(t) //generate a prompt to validate the correctness of thought t
API for Parser (extensible)¶
注：图中蓝色部分包含架构概述，绿色部分列出API¶
Example prompts and the Graph Reasoning State for the sorting use case¶
注：红色部分包含示例提示以及GRS和相关操作，具体是归并排序（先把list array，分解成sub array，sort后进行合并）任务的prompt的示例
思维算法 Algorithm of Thoughts (AOT)：一种用DFS/BFS示例解决问题的方法¶
论文名称：Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
论文地址：https://arxiv.org/abs/2308.10379
论文 Github：https://github.com/kyegomez/Algorithm-Of-Thoughts
1. 为什么需要思维算法 Algorithm of Thoughts (AOT)?¶
• Standard Prompting 旨在直接回答
· CoT 给出了最终解决方案的连续步骤
2. 思维算法 Algorithm of Thoughts (AOT) 思路是什么?¶
AoT的上下文示例与CoT不同，它集成了搜索过程，由标记“1”、“...”、“突出显示3”作为指导问题集子树探索的“第一次运算”8644。为了清楚起见，只显示了一个上下文中的示例，重点放在第三子树探索上。AoT产生预期的搜索步骤（例如子树探索'5。11+1'），并评估任何进展的潜在后续步骤找到一个解决方案或返回到另一个可行的子树。
3. 思维算法 Algorithm of Thoughts (AOT) vs 其他 COT 的区别?¶
• AOT vs COT:
·COT：为链式结构，也就是问题答案只能由一条固定路径得到；
· AOT：为有向图结构中寻找最优路径的过程；
• AOT vs TOT:¶
· TOT：对节点进行剪枝，然后推理过程是一个树的结构；
· AOT：利用示例的方式模仿DFS或者BFS，能够激发LLM的能力（论文说在GPT4上有很好的效果，估计对LLM本身的能力要求比较高），得到更好的结果：
思维链 Chain-of-Thought (COT) 有哪些 应用场景? Task 1: 复杂任务求解¶
从论文给的一些数学和算法的求解示例可以看出，这些复杂的思维链的prompts工程是为了增强模型在复杂推理任务的准确性，所以显而易见，后续的一个很大的应用方向是解决复杂的数学问题，组合优化问题等等，至于能不能求解NP-hard问题，找到一些启发式的算法，还有待探索。
Task 2: 增强特定任务可靠性¶
除了解决LLM在复杂任务上的性能问题外，还有一个方向就是提升已有任务的准确性，比如现在检索+生成的问答方式只能达到80%的准确率了，通过应用思维链的方式+限制条件，一下子能够提升准确率到90%。这对于一些追求LLM稳定性结果输出的场景具有重要的作用。
思维链 Chain-of-Thought (COT) 有哪些局限性？¶
CoT，AoT，ToT都需要你自己写instructions和in-context examples，这些都属于prompt工程的范畴，但是人工的寻找某些任务的最佳prompts就比较费脑子，因此自动化prompts工程会是未来一个潜在的方向，来弥补现在对复杂任务需要这种细粒度的prompt工程的弊端。
知识星球
