Quill as a DLL: Sinks Between DLLs.

[Shawnthebro]

Hello!

The engineers at my work and I really love Quill. We have a need to use Quill across DLLs. I have built Quill as a DLL and it works when using it in one DLL. If I try to LOG from two different DLLs using the same logger, it will work until the memory is freed (when calling FreeLibrary). I believe this is because the Sinks are made with a shared_ptr. Shared pointers are difficult to work with across DLLs. Is there a way to get a console as a raw/standard pointer or another way to handle this so we can keep using Quill in this way without crashing when releasing memory? Thank you for your insight!

[odygrd]

Hey, the sinks are managed via shared_ptr, ensuring that when all Logger objects holding a reference to a sink are removed, the sink is automatically destructed, and any underlying files are closed properly.

Would you mind sharing a quick example of your setup, including the use of FreeLibrary? This would help me reproduce the issue on my end, and I’ll try to look into finding a fix

[Shawnthebro]

Thank you so much for the quick reply! I can't share the specific behavior due to it being a work project, but I would be happy to recreate it and see if I can run into the same issue on my machine. For a general overview I can share right away, it goes like this.

There is a Launcher.dll that is loaded by the main application (.exe). The Launcher goes through and launches all modules (other DLLs). We have some modules for running tests, others for sending messages over a network, so on and so forth. We have another DLL that is very simple and basically is the exact code from quill_wrapper_shared in the examples folder. All modules and the Launcher use an extern logger variable matching the logger from that Quill DLL.

In the Launcher, we call setup_quill() and set the log level. In the Launcher and all modules, we use LOG_TRACE to log out whatever relevant data we want. At the end of the program before exiting the application, we call FreeLibrary. This releases all module DLLs first (FreeLibrary is called on each module), then the Launcher DLL (FreeLibrary is called on the Launcher). When FreeLibrary is called, there is a crash in the shared_ptr decrement code. If we remove the log call from all but one of the DLLs (such as only leave Quill logs in the Launcher), no crash will occur when calling FreeLibrary. For an example, if we had the LauncherDLL and a TestModuleDLL with both of them using a LOG_TRACE call, then FreeLibrary would crash.

I've also tried manually stopping Quill logs in a few ways, but I have not be able to solve the problem. Although I am pretty familiar with the architecture of the system, I am new to using Quill, so I apologize if we are doing something incorrectly and I have not caught it. Thank you so much for taking the time to respond.

[odygrd]

Thanks for the quick follow-up. Yes, it would be really helpful if you could recreate the issue with a simple example that I can use to reproduce it on my end. I’ve attempted to replicate the behaviour in this example here:

https://github.com/odygrd/quill/blob/shared_lib_example/examples/shared_library/example_shared.cpp

However, I haven't been able to reproduce the crash yet. When you have some time, could you take a look at the example and let me know if there's anything I should adjust to better reflect your setup? Any additional insights would be much appreciated!

[odygrd]

I’ve put together another example that attempts to mirror what you described, but it's not fully working as expected yet. It doesn't seem to be the same issue, and I think there might be something off with my DLL setup in this case.

If you have time, it might be helpful to take a look at this example and suggest any modifications to make it better reflect your setup:

https://github.com/odygrd/quill/tree/shared_lib_example_2/examples/shared_library

Also are you linking dynamically ‘/MD’ ?

https://learn.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-170

[odygrd]

Hi, I’ve updated the example to better mirror what you described, and it’s now at least running on my PC. When you get a chance, please take a look and see if you can modify it to reproduce the issue:

https://github.com/odygrd/quill/tree/shared_lib_example_2/examples/shared_library

[Shawnthebro]

I really can’t thank you enough! This example is very close to how it is set up for us. I am working today and have some other things to take care of, but I will see what else I can learn about the issue.

Quill is setup in the launcher, then the modules are created and freed in the launcher as well. At the end of the Launcher, stop_quill is called.

However, FreeLibrary is causing a crash when the Library DLL is freed at the end of the application.

I don’t believe we are dynamically linking ‘/MD.’ I would need to clarify on this and will try to do so today. Thank you again! I am grateful!

Update: I just saw you have the example_shared which mimics our application as well. Yes, you’ve nailed the setup and how we are using it!

[odygrd]

I’m not entirely sure if /MD linking is related—I came across that suggestion somewhere but don’t have much experience working with DLLs.

One thing I did in the example was to call flush_log in DllMain for each module, as it was crashing when a module was freed with pending logs.

quill/examples/shared_library/test_module/include/test_module/test_module.cpp

Line 21 in a731232

global_logger_a->flush_log();

Maybe give that a try and see if it helps. If not, feel free to tweak the example to better match your setup.

[Shawnthebro]

This was it. Thank you so much for your time. You are the greatest. We love Quill and I’m very happy we can continue to use it. Please let me know if you need anything or if I can pay you back for your assistance.

[odygrd]

Hey, thanks for confirming! I believe I can see the issue. Quill creates a ThreadContext instance as thread_local. When the thread goes out of scope, it doesn't delete the instance; instead, it invalidates it and the instance is held as shared_ptr by the Backend thread

The backend logging thread then detects this invalidated ThreadContext and attempts to empty the log queues before deleting it. It appears that Windows runs the DLL on its own thread, meaning it has its own ThreadContext instance.

When you call FreeLibrary, the memory holding that instance is released, but the backend worker still tries to access checking for empty log queues. This leads to a crash because it's now pointing to invalid memory.

I've noticed that within DllMain, the thread_local destructor has run and at that point ThreadContext is invalidated. We need to ensure that the backend thread has removed the ThreadContext instance from it's cache before we exit DllMain.

Currently, there's a race condition, which works because of the sleep_duration in log_flush(). To be a bit safer, you can implement the following:

#include "quill/Backend.h"

case DLL_PROCESS_DETACH:
  global_logger_a->flush_log(); 
  quill::Backend::notify(); 
  std::this_thread::sleep_for(std::chrono::milliseconds{500});

I will add a fix to the master branch to handle this case.

[Shawnthebro]

Excellent! Thank you for the detailed explanation.

[odygrd]

This is fixed in commit 5567063 and latest release 7.2.2

You still have to call flush_log() but it should now be safe from race conditions, also no additional function calls should be necessary

case DLL_PROCESS_DETACH:
    global_logger_a->flush_log();
    break;

[Shawnthebro]

We are using it now and it works like a charm! Thank you again for the quick responses and amazing help. We are grateful!