Real World Prompting Feedback

[thenameless7741]

I recently converted Real World Prompting tutorial to TypeScript to make it more accessible (and for my personal learning and enjoyment). Along the way, I made a few adjustments to the content, so I just wanted to share these changes with the course creator.

1. API Parameters

• The tutorial is still referencing the old sonnet model (claude-3-sonnet-20240229).
• The temperature param is not set, even for tasks that could benefit for it (e.g. summarization).

2. Prompt Adjustments

As a result of adjusting the params in 1), I'm seeing way fewer hallucinations and out-of-character behavior (e.g. an assistant referencing <context>), but I noticed some areas where performance declined. For example:

• In chapter 4, I had to explicitly instruct Claude to output the <json> tag after "Generate a JSON object with the following structure". Without that, Sonnet 3.5 would skip wrapping its response in tags (which might be preferable in most cases).
• Also in chapter 4, when dealing with incomprehensible (e.g. "blah blah blah") and empty transcripts, Sonnet 3.5 would fail to output a JSON object. Instead, it responded with a long explanation asking for a valid transcript. I solved this by adding two additional examples to the prompt.

If you'd like more details on these changes, please let me know. For reference, here's the project I've been working on.

Lastly, I appreciate the effort put into creating this course and I'm looking forward to future updates!