GoBackAsync() not working

[mnxamdev]

I'm using the INavigationService by injecting it into my own service and handling navigation that way. The reason why we're doing that is historical but mainly because we do additional logging when navigating so we would like to keep the responsibility for that in one class. Navigation works but when I use GoBackAsync() it doesn't go back. If I look into the NavigationStack there is always only one item. Setup is below. I hope we can just inject INavigationService and use it from anywhere otherwise I'd have to rewrite a lot of code for navigation. In the case below ViewB is not a view with a view model at all because ViewB is rendered entirely using a custom renderer and from there we use our custom NavigationManager to navigate. Thanks in advance.

Class we're injecting INavigationService into:

public class NavigationManager
{
    public NavigationManager(IMyLogger logger, INavigationService navigationService)
    {
        _logger = logger;
        _navigationService = navigationService;
    }
}

MauiProgram.cs

prism.OnAppStart(navigationService => navigationService.CreateBuilder()
                        .AddSegment<MainMasterDetailPage>()
                        .AddSegment<MasterDetailContentNavigationPage>()
                        .AddSegment<MyMainView>()
                        .NavigateAsync(HandleNavigationError));

Sometimes we want to be able to go back from ViewB so we do ViewX > ViewB where ViewX can be any View so I'm doing this anytime we want to go to ViewB:

await _navigationService.CreateBuilder()
                        .AddSegment<MasterDetailContentNavigationPage>()
                        .AddSegment<ViewB>()
                        .NavigateAsync(HandleNavigationError);

However, GoBackAsync() isn't working for me:

public async Task GoBack()
{
    await _navigationService.CreateBuilder()
                       .GoBackAsync();
}