Skip to content

Commit 73e91cf

Browse files
committed
Add README and related sample files
1 parent 398a778 commit 73e91cf

File tree

4 files changed

+53
-1
lines changed

4 files changed

+53
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ RAG_ONLINE_RETRIEVED_TOPK: 1 # The topk for the online retrieved documents
142142
Adjust `RAG_ONLINE_SEARCH_TOPK` and `RAG_ONLINE_RETRIEVED_TOPK` to get better performance.
143143

144144

145-
#### RAG from Previous-Experience
145+
#### RAG from Self-Demonstration
146146
Save task completion trajectories into UFO's memory for future reference. This can improve its future success rates based on its previous experiences!
147147

148148
After completing a task, you'll see the following message:
94.7 KB
Loading

record_processor/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
2+
# Enhancing UFO with RAG using User Demonstration
3+
4+
UFO can learn from user-provided demonstrations for specific requests and use them as references in the future when encountering similar tasks. Providing clear demonstrations along with precise requests can significantly enhance UFO's performance.
5+
6+
## How to Prepare Your Demostration ❓
7+
8+
### Record your steps by Microsoft Steps Recorder
9+
10+
UFO currently support study user trajectories recorded by Steps Recorder app integrated within the Windows. More tools will be supported in the future.
11+
12+
**Step 1: Record your steps**
13+
14+
You can follow this [official guidance](https://support.microsoft.com/en-us/windows/record-steps-to-reproduce-a-problem-46582a9b-620f-2e36-00c9-04e25d784e47) to record your steps for a specific request.
15+
16+
17+
**Step 2: Add comments in each step if needed**
18+
19+
Feel free to add any specific details or instructions for UFO to notice by including them in comments. Additionally, since Steps Recorder doesn't capture typed text, if you need to convey any typed content to UFO, please ensure to include it in the comment as well.
20+
<h1 align="center">
21+
<img src="../assets/record_processor/add_comment.png"/>
22+
</h1>
23+
24+
25+
**Step 3: Review and save**
26+
27+
Examine the steps and save them to a ZIP file. You can refer to the [sample_record.zip](./example/sample_record.zip) as an illustration of the recorded steps for a specific request: "sending an email to [email protected] to say hi."
28+
29+
30+
## How to Let UFO Study the User Demonstration ❓
31+
32+
33+
Once you have your demonstration record ZIP file ready, you can easily parse it as an example to support RAG for UFO. Follow these steps:
34+
35+
```console
36+
# assume you are in the cloned UFO folder
37+
python -m record_processor -r <your request for the demonstration> -p <record ZIP file path>
38+
```
39+
Replace `your request for the demonstration` with the specific request, such as "sending an email to [email protected] to say hi."
40+
Replace `record ZIP file path` with the full path to the ZIP file you just created.
41+
42+
This command will parse the record and summarize to an execution plan. You'll see the confirmation message as follow:
43+
```
44+
Here's the plan summarized from your demonstration:
45+
(1) Input the email address '[email protected]' in the 'To' field.
46+
(2) Input the subject of the email. I need to input 'Greetings'.
47+
(3) Input the content of the email. I need to input 'Hello,\nI hope this message finds you well. I am writing to send you a warm greeting and to wish you a great day.\nBest regards.'
48+
(4) Click the Send button to send the email.
49+
Would you like to save the plan future reference by the agent?
50+
[Y] for yes, any other key for no.
51+
```
52+
Press `Y` to save it into its memory for furture reference. A sample could be find [here](../vectordb/demonstration/example.yaml).
486 KB
Binary file not shown.

0 commit comments

Comments
 (0)