Per-gptel-buffer contexts

[orge-dev]

Describe the solution you'd like
I like to define sets of files/buffers as context, and it would be convenient to have a gptel buffer correspond to a particular set of files as context. So I could have * Claude (No Context) * buffer, * Claude (Project A1) * buffer, *Claude Project A2* ,etc.

Describe alternatives you've considered
An easy way to clear the whole gptel context, plus user defined (gptel-add-file file) functions would serve a similar purpose. I'm not sure how to easily remove all items currently in the context.

Additional context
Thanks for creating and maintaining gptel!

[karthink]

I think I understand what you mean. Let's call it the "buffer-local context sets" feature, because you can have independent sets of context. This feature makes sense in the abstract, but not in the details. Here's why I think it's under-specified:

1. Suppose you visit some buffer, select a region and run gpel-add to add it to the context.

   • Which gptel buffer's context (context set) should this be added to?
   • How about when there is no chat buffer? gptel is usable in any buffer, and most of the time I don't use a chat buffer.

2. Same question as 1, but for when you run gptel-add in a dired buffer to add a file.

3. Suppose you run gptel-add-file, either from the transient menu or via M-x. This command can be run in any buffer. Which context set should it be added to?

I imagine what you're describing is actually "project-local context sets", not "buffer-local context sets". Now each project (as managed by project.el or projectile) has an independent gptel context, and any of the above actions 1-3 will add the region/buffer/file to its project's context. In this case, what happens when you run/want to include a region/buffer/file in project A's context that does not belong to project A?

[metachip]

Thanks for the reference to this issue @karthink, hcere's my thoughts on the questions you posed to @orge-dev

I have two suggestions.

Multiple Context Buffers

I agree the concept here is "project local" but I would not recommend making it dependent on projectile or similar. Instead, one solution might be to maintain a list of context buffers, and allow them to be selected and associated with chat (or other) buffers as required. There would always still be one global context buffer. It would be the default when a default is required.

For example, when one invokes an action that uses the context, if the context that should be used is indeterminate, use global context. If the context can be inferred, for example, because it has been previously associated with a chat buffer, then use the context associated with that chat buffer instead of the global context. I imagine another widget at the top of the chat buffer would make this selection easy.

If an action is taken from an arbitrary buffer, to add new context, pop open a list of context buffers to enable the choice of target, prior to taking the action to add the new context to it. The default action, if nothing is otherwise selected, would be to add the new context to the default context buffer.

Likewise in the gptel menu, one might allow the selection of what the current default context buffer is, so that when using gptel in an arbitrary buffer, one has the option to change the default without deleting and manually rebuilding to the new context.

Context Links

Instead of including context by way of separate buffers, one could instead use links. This would especially apply when one is using org mode to have a dialogue with a model but need not be limited in this way. I've outlined my thinking on this further here.

#481

The bottom line for me is that context is fundamentally important, especially given the extremely flexible way you've designed gptel to work "anywhere" across emacs. Being able to easily associate the "right context" with myriad concurrent tasks that I'm working on across emacs, whether in chat buffers or anywhere else, would be fantastic.

[2024-11-24 Sun 18:25]

[karthink]

This proposal for multiple context buffers is going to be confusing to most users. I would prefer gptel-add and co to be a one-shot interaction -- every additional decision the user has to make when taking a gptel action will annoy them, as evidenced by past threads like #374 (among others). If we can find a way to make gptel-add Do The Right Thing in all situations I wouldn't mind working on this.

I like the context link proposal in #481 --it's a very simple mental model that I expect users won't have trouble following. let's discuss this in the dedicated thread.

[dwrz]

I am a new user to gptel -- thank you for all your work on it. I have been really happy with my experience so far, but this issue is the first limitation I felt I ran into. Would it be possible to just have a singular buffer local context?

I may have a chat going on for one project, another for another project, another for a topic, and then I may throw a quick question into scratch, and so on. When switching across these, if I need to use context, I have` to dump the context, and then rebuild it -- and in some cases that can be quite a few files. For me, having the context be buffer local would be sufficient, but perhaps I am not thinking about all the use cases -- the only one I can think of is jumping to different files to ask for a refactor / rewrite; you'd have to still rebuild your context.

The only other thing I can think of is being able to assign contexts to variables, and then pull them up by history the same way we can with past directives.

[karthink]

@dwrz I feel the pain of dumping and rebuilding the context all the time. I just don't know how to resolve it.

Buffer-local context does not work because it's ambiguous which buffer's context is being added to when you run gptel-add.

As suggested by @metachip, one way to solve this is to simply ask the user which context set they want to add to. My experiences with this kind of branching in gptel suggests that it's going to be more annoying than useful, and quite confusing too, leading to doubts like:

• "Wait, which context set is active right now?"
• "Did I add the file to the right context set?"
  It also requires gptel to maintain an internal registry of all active context sets, invalidate this registry etc.

I like the idea of using Org/markdown links to add to the context, as discussed in #481, as the context set automatically becomes buffer-local, transparent and easily editable. But that has other issues, including that you can't have "live" buffer regions in the context.

[metachip]

This proposal for multiple context buffers is going to be confusing to most users. I would prefer gptel-add and co to be a one-shot interaction -- every additional decision the user has to make when taking a gptel action will annoy them, as evidenced by past threads like #374 (among others). If we can find a way to make gptel-add Do The Right Thing in all situations I wouldn't mind working on this.

Agreed. It needs to be DWIM as far as possible.

1. Can you not just use the prefix argument C-u ?

   That is, always add/remove context from the global context, if gptel-add* called directly or via the transient menu.

   If called directly with a prefix argument, it pops open the list of available context buffers and allows one to create a new one, just like gptel does now with respect to chat buffers and their names.

   Existing users will be none the wiser and gptel-add remains a one-shot operation.

2. Add another widget to the top of chat buffers to indentify which context is being used.

   This would only show if the context is not the default.

   You could merge it into the context widget you already have, which in addition to reporting the files and regions, says the name of the context too.

@dwrz

I have` to dump the context, and then rebuild it -- and in some cases that can be quite a few files.

For me this is at the very heart of the problem that we are discussing here.

1. The idea of context, local to a chat buffer as you suggest is one solution.

2. The association of one, of several named context buffers, to a chat buffer, is another. This is my preferred solution.

3. Context that appears in a buffer at the moment, could, for any org mode buffer, be put in a drawer called (say) :CONTEXT: (analogous to the special case of :PROPERTIES: for properties.

All that said, I think links in a buffer may be a better solution per #481.

I'll continue my thoughts in that thread.

[karthink]

Can you not just use the prefix argument C-u ?

There is quite a bit more to it than that. For example, it causes ambiguity in the indication of the current context in the transient menu and in constructing the context inspection buffer (which context set is looked up?). There's no longer a single source for the *gptel-context* buffer, compounding the update problems discussed in #482.

I also think you are underestimating the amount of confusion an opaque many-to-one associative map from buffers to context sets will cause when using gptel. It will be something you have to keep double-checking before sending requests, especially from non-chat buffers ("Am I using the right context set here?"). In this specific aspect having "project-local" context sets works a little better -- this is also a many-to-one map but people are already used to thinking this way about collections of files/buffers.

[metachip]

I also think you are underestimating the amount of confusion an opaque many-to-one associative map from buffers to context sets will cause when using gptel.

Fair enough. I think links are better anyway because they would be more explicit in terms of the context one is working with.

I have some thoughts about the questions you have raise about links which I will answer in that ticket shortly.

[dwrz]

@karthink -- thank you! I'm starting to see that there's much more to it than what a first impression suggests.

What about the possibility of using variables to store contexts?

For example:

(setq my-context-a '("/home/user/some-file" "/home/user/some-file-2" "/home/user/some-file-3"))
(setq my-context-b '("/etc/foo" "/etc/bar" "/etc/baz"))
(setq gptel-contexts '(my-context-a my-context-b))

Then in the transient menu, either let the user build a context as is currently possible, or load one from a submenu, as is possible for directives. Could something like that work?

[nlevnaut]

Thinking through the UX for multi-context, perhaps this would work without being too confusing for the default case:

• By default just provide 1 "context group". Existing add/remove commands stays the same.
• Let the user create new named context groups. (not per gptel buffer, just global)
• Switch between the currently-in-use context group (globally) with a gptel-context-switch command.

[karthink]

Thinking through the UX for multi-context, perhaps this would work without being
too confusing for the default case:

• By default just provide 1 "context group". Existing add/remove commands stays
  the same.
• Let the user create new named context groups. (not per gptel buffer, just
  global)
• Switch between the currently-in-use context group (globally) with a
  gptel-context-switch command.

Because context state is stored in buffer overlays, looking at buffers will not give you an idea about which regions are included in the current context. This can get confusing very quickly.

However it should work fine for personal use, if you're aware that the source of truth is what the transient menu (or context inspection buffer) says:

(defvar gptel-context-groups nil)

(defun gptel-context-switch (group-name)
  "Switch to or create gptel context GROUP-NAME."
  (interactive
   (list (completing-read "Context group (switch or create): "
                          gptel-context-groups)))
  ;; Save current group if required
  (unless gptel-context-groups
    (push (cons "default" gptel-context--alist) gptel-context-groups))
  ;; Switch to or create group
  (if-let* ((group (cdr-safe (assoc group-name gptel-context-groups))))
      (setq gptel-context--alist group)
    (setf (alist-get group-name gptel-context-groups
                     nil nil #'equal)
          (copy-tree gptel-context--alist))))

Call M-x gptel-context-switch to switch context groups.

[ultronozm]

After reading this and related threads, I made a "local context" prototype https://github.com/ultronozm/gptel-local-context.el that I figured I'd share at least for the sake of the discussion. I don't know to what extent it gels with the design ethos, and would be happy to help contribute it (or not) accordingly. You can try it out by loading the package file and applying the following diff to gptel.el:

@@ -2063,8 +2063,7 @@ be used to rerun or continue the request at a later time."
          (gptel--system-message
           ;; Add context chunks to system message if required
           (unless (gptel--model-capable-p 'nosystem)
-            (if (and gptel-context--alist
-                     (eq gptel-use-context 'system))
+            (if (and (eq gptel-use-context 'system))
                 (gptel-context--wrap (car directive))
               (car directive))))
          (stream (and stream gptel-use-curl

We don't touch gptel-add -- it remains specific to global context. We adjust local context for the current buffer in the transient menu. We take "local context" to be a buffer-local list of strings, interpreted as buffers, files or functions. We disallow "buffer-local context overlays", which would probably be confusing; plus, we can always use indirect buffers.

Notes:

• Independent of this discussion, functions can be a useful source of context, see for instance https://github.com/ultronozm/ai-org-chat.el?tab=readme-ov-file#ai-planner.
• Local context "presets" or "groups", intended for use across multiple buffers, could be added on top of this.
• Tools can also have useful buffer-local settings.

[metachip]

@ultronozm Very interesting.

The UI you have created to add all sorts of context types looks great. I'll give it a go.

What follows are my initial thoughts in light of what you have done, other things discussed in this tread and some others: Context links (#481), Context management (#484) and Context buffer functionality (#482).

I still have an open mind about the best way to manage context. Buffer local, project local or explicitly named. They all make sense for different use cases. However, there are some challenges when integrating all this into the workflow of GPTel. In particular, ambiguity about which context is active, which context to add to or delete from and how to see what a context comprises. This is especially true because it needs to handle both chat and non-chat buffer scenarios.

I think this is really a UI question more than anything else. I agree with your suggestion about the use of indirect buffers instead of overlays. I don't think it is necessary to try and show all of the context in a single buffer as GPTel tries to do now. Lists of references to files, regions or other entities which when activated open in an indirect for regions or buffers or new buffer if the item is from a file should be enough.

As for knowing which context applies, your solution, making context local to the buffer, is simple and straightforward.

That said, I think a combination of explicitly named contexts, a default context, and means by which one can choose which one to apply at each invocation of gptel-send is better. This is because GPTel is designed to "dance around emacs" rather than remain focused on a single buffer. Chat buffers are the exception of course but I think @karthink is very keen to apply the KISS principle as much as possible, with the lightest of touches on any other mode, function or feature offered by emacs.

I think we can integrate something along the lines of what you have developed, but within a more flexible framework of named contexts, where the context that is applied depends on a set of simple rules which intuitively default back to what GPTel does now.

• At gptel-send time
  • An optional quick context selector
  • Default to buffer-local setting if exists
  • Fall back to global default context
• Persistent association ideas
  • Buffer-local variable to store preferred context
  • In the PROPERTIES drawer (in org mode files)
  • Project-level context preferences

There would also need to be simple means by which context can be managed. We need to create contexts, edit them, add/remove files, regions and metadata. Clear, merge, copy, save and restore.

How one does this presumably means we'd create some context editing mode that opens a context in a buffer and gives us tools to manage it. I have fleshed out a proposal for org mode buffers, using links #481, which extends the metaphor of media links that are already part of GPTel to enable context to be created, edited and deleted by simply embedding org mode links in text.

Anyway, my two cents for now, I will give it some more thought.

[metachip]

I forgot to add. I think context should include more than just references to regions, files and media. It should also include what GPTEL already manages; the selected model, the back end, the system prompt.