Could not load file or assembly 'System.Management.Automation' after update to .NET 7 (3.16.x)

[KJoreth]

After updating our project to .NET 7, NUnitConsole won't run any of our tests via Windows PowerShell.
After running .\nunit3-console.exe("tests.dll") we get an error:

Unable to load one or more of the requested types.
Could not load file or assembly 'System.Management.Automation, Version=7.3.1.0

Even though running the same tests with dotnet test runs just fine.
We have tried to run it with NUnitConsole 3.16.0 / 3.16.1 and 3.16.2

Scripts are run on PowerShell 5.1.19041.1237 and Windows 10 (build 19043.1288). We did try to use the newest version of PS as well but without any success.

Nuggets in out project which may have some influence:

"Microsoft.NET.Test.Sdk" Version="17.4.1"
"Moq" Version="4.18.4"
"MSTest.TestAdapter" Version="3.0.2"
"MSTest.TestFramework" Version="3.0.2"
"NUnit3TestAdapter" Version="4.3.1"
"Microsoft.PowerShell.SDK" Version="7.3.1"

Log file you can find here:
InternalTrace.6200.log

Running the script with --inprocess flag returns:

NUnit.Engine.NUnitEngineException : The NUnit 3 driver encountered an error while executing reflected code.
  ----> System.InvalidCastException : Unable to cast transparent proxy to type 'System.Web.UI.ICallbackEventHandler'.

--NUnitEngineException
The NUnit 3 driver encountered an error while executing reflected code.
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Drivers\NUnit3FrameworkDriver.cs:line 165
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Drivers\NUnit3FrameworkDriver.cs:line 85
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Runners\DirectTestRunner.cs:line 136
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage() in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Runners\DirectTestRunner.cs:line 126
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded() in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Runners\DirectTestRunner.cs:line 235
   at NUnit.Engine.Runners.DirectTestRunner.RunTests(ITestEventListener listener, TestFilter filter) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine.core\Runners\DirectTestRunner.cs:line 184
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine\Runners\MasterTestRunner.cs:line 472
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter) in C:\projects\nunit-console\src\NUnitEngine\nunit.engine\Runners\MasterTestRunner.cs:line 150
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter) in C:\projects\nunit-console\src\NUnitConsole\nunit3-console\ConsoleRunner.cs:line 197
--
InvalidCastException
Unable to cast transparent proxy to type 'System.Web.UI.ICallbackEventHandler'.

   at NUnit.Framework.Api.FrameworkController.LoadTestsAction..ctor(FrameworkController controller, Object handler)

On NUnitConsole 3.15.2 and .NET 6 everything works. NUnitConsole was installed via zip file.

[mikkelbu]

Regarding the --inprocess flag, then I think it is the same problem as in #1208 (comment) (that the console is compiled under .NET framework and the tries to load a .NET Core test dll).

Regarding the original issue, then I guess it is due to conflicting engine versions (the console comes with one version and the NUnit3TestAdapter comes with another). If you remove the reference to NUnit3TestAdapter can you then run the tests in the console?

[KJoreth]

Thank you for your answer.

Unfortunately removing reference to NUnit3TestAdapter and not using --inprocess flag did not change anything, we still get the same error Could not load file or assembly 'System.Management.Automation

[KJoreth]

@mikkelbu Removing reference to Microsoft.PowerShell.SDK from the project allowed us to run tests via NUnitConsole. Sadly we still have no idea why is that the case. Any ideas?

[mikkelbu]

@KJoreth I'm sorry that I missed your comment, but I've no idea of why it does not work

[accentsoftware-simonl]

I'm getting similar issues with .NET 7.

NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Tuesday, 7 March 2023 5:17:33 PM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    Portals\Insyte.IBL\InsyteWeb\Libs\InsyteWeb.IntTests.Core\bin\Release\net7.0\win-x64\InsyteWeb.IntTests.dll

Test Filters
    Where: cat!=UnitTest and cat!=NotForNightlyBuild


Errors, Failures and Warnings

1) Invalid : D:/SCC/InsyteBlinds/Portals/Insyte.IBL/InsyteWeb/Libs/InsyteWeb.IntTests.Core/bin/Release/net7.0/win-x64/InsyteWeb.IntTests.dll
Unable to load one or more of the requested types.
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Security.Claims, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Security.Claims, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Could not load file or assembly 'System.Linq.Expressions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Cannot load a reference assembly for execution.
  ----> Could not load file or assembly 'System.Security.Claims, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL'. Reference assemblies cannot be loaded for execution. (0x80131058)
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.
  ----> Cannot load a reference assembly for execution.

Run Settings
    DisposeRunners: True
    WorkDirectory: D:\SCC\InsyteBlinds
    MaxAgents: 1
    NumberOfTestWorkers: 0
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETCoreApp,Version=v7.0
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    TargetRuntimeFramework: netcore-7.0

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2023-03-07 06:17:33Z
    End time: 2023-03-07 06:17:34Z
    Duration: 0.589 seconds

I have tried modifying tools\agents\net7.0\nunit-agent.runtimeconfig.json to use version 7.0.2 that I have installed but it makes no difference. Also tried Microsoft.WindowsDesktop.App with both 7.0.0 and 7.02.

[Edit: added below but then discovered it is actually because I dropped the reference to Microsoft.NET.Test.Sdk thinking it was not needed]
I just discovered dotnet test has stopped working too - reverting NUnitTestAdapter to 4.3.1 from 4.4.2 fixes it. This was the error and also a dependency resolving issue:

D:\SCC\InsyteBlinds>dotnet test Portals\Insyte.IBL\InsyteWeb\Libs\InsyteWeb.UnitTests.Core\bin\Release\net7.0\win-x64\InsyteWeb.UnitTests.dll --filter TestCategory=UnitTest --configuration Release
The following arguments have been ignored : "--configuration Release"
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) 'D:\SCC\InsyteBlinds\Portals\Insyte.IBL\InsyteWeb\Libs\InsyteWeb.UnitTests.Core\bin\Release\net7.0\win-x64\InsyteWeb.UnitTests.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (InsyteWeb.UnitTests.deps.json) was not found:
    package: 'AutoFixture', version: '4.17.0'
    path: 'lib/netstandard2.0/AutoFixture.dll'
. Please check the diagnostic logs for more information.

Test Run Aborted.

[OsirisTerje]

@accentsoftware-simonl @KJoreth I think this might be related to #1324.

In your bin folder, do you have a file there named Microsoft.Extensions.DependencyModel ? Check the version properties of that file, and if it says 3.1 then that is probably the issue.

From #1324 you can see the issues we noted on the TestAdapter, and we had to revert back to 3.15.2.

@accentsoftware-simonl I see you also use dotnet test. In that case, remove the NUnit.Engine from your refs, and ensure you have adapter 4.4.2 there. Then it should work.

Also, note that running with NUnit.Engine version 3.16.X and any version of the adapter (which you need for dotnet test) is incompatible.

[accentsoftware-simonl]

Thanks @OsirisTerje. I am trying to switch from dotnet test to nunit console because I have older .Net 4.6.2 tests running in parallel (via MSBuild parallelism) that I want to run in parallel with these tests on our build server.

[BTW, part of why I am doing that is that there doesn't seem to be a way with nunit's 2 parallelism mechanisms to group sets of classes (across multiple assemblies) which can run in parallel with other groups of classes but not within themselves.]

dotnet test works with adapter 4.4.2 and also NUnit.Console 3.16.3 package referenced.

Downgrading console to 3.15.2 gives similar but different assembly issues

Same result also downgrading adapter to 4.3.1

[OsirisTerje]

@accentsoftware-simonl Can you provide a repro for the latest here? I don't see any reason for adapter 4.4.2 not to be able to load assemblies, given that there are no NUnit.Console or NUnit.Engine version 3.16.X present.

If you're going to be using NUnit.Console 3.16.X you need to - during build - remove what comes from the adapter package - for that test scenario.

PS:
About running assemblies in parallel, there are multiple ways to do that with e.g. Azure pipelines, Github actions or CircleCI. Within an assembly you can use the NUnit parallelization attributes to control how to run them internally. I still don't see why you would need NUnit.Console to do that. It will just add pain to your process. Just IMHO......

[accentsoftware-simonl]

@OsirisTerje I have some higher priorities at the moment but should be able to get back to this in the next few weeks. I do notice one test assembly out of 3 works fine so it may have something to do with indirect dependencies - I think there are a few .NET Standard assemblies referenced - I remember seeing something about .NET Standard not being supported so would that be it?

[OsirisTerje]

@accentsoftware-simonl Net Standard should work fine as SUT.

[undron]

This looks fixed in 3.16.3 release

[MichalMucek]

I confirm. It's fixed.

[accentsoftware-simonl]

@OsirisTerje @MichalMucek
My issue was occurring in 3.16.3 but I haven't had time to get back to it.
If I do get back to it I can raise a new ticket though.

[OsirisTerje]

@accentsoftware-simonl Sorry if I closed it too soon, feel free to reopen this if you still see the issue.
@garrystalin You downvoted the fix comment from @undron . Does that mean you do have the issue?

To all: We still don't have any repro for this, so if multiple people have this issue, it should be possible for one of you to provide a small repro. We DO need that in order to work out a fix :-)