[FEAT] Re-Analysis Before Voting

[NeonClary]

Objective

Implement re-analysis of input including consideration of other participant's comments before voting

Initial Implementation Requirements

• Context sharing, revised prompting...

Other Considerations

• Bohdan may be able to implement some of this? Or it may be entirely V3.
• We would like to enable a true discussion. This issue meets the minimum requirement for this, but adding re-analysis before voting, with the comments of the other LLMs as context.
• To do this we will need to select the messages which should be included as context in each submission to the LLM participants. Challenges include:
• The insertion of messages from humans having side discussions with LLM participants while the main discussion is going on.
• That both our LLMs and the commercial ones currently support only two conversational entities, the user and the LLM. Our conversation contains many entities, and choices need to be made on how to work within that limitation.
  One way to adapt to sharing the context with multiple participants in a conversation is to insert some additional contextualizing text just before the conversation history and after the system prompt. An example might be "...You have been participating in a CCAI group discussion, you may have already commented, now you need to respond again. Here is the conversation so far:"
  Later, an additional LLM could be used to refine this contextualizing text to be more specific the LLMs participation in the preceding conversation. For example, "...apparently you have already commented twice in the discussion round, and now it's time to vote…"
• For V3 it would also be desirable to enable a round of re-comment after re-analysis, and multiple rounds of comment and analysis, to build a robust discussion and decision-making process.
• Perhaps the LLMs could be given a preliminary count of votes before their second (or in the future, third or fourth) response, and led to discuss in various ways. Prompts might include "this answer was most popular, not the one you chose. Consider these reasons why it may be better than your chosen answer and decide if you should change your vote." Or, "your answer is the most popular so far, this answer is the second choice, offer reasons why it is not a good choice". Or to relate it back to Perplexity, the context of the other comments could be applied to re-analyze the Perplexity of their own response and then that of others. This area will need a lot of thought.