Modify `Config` without creating new `CachingClient`

[macovedj]

At the moment, the CachingClient, which is created using a Config can not have its config mutated once it has been created. Thus if say, an override were to be added for a package or namespace, a new client needs to be created, rather than just modifying the config for the existing one. Instead, methods could be added to CachingClient that enable updating the config.