UI: Update profile encoder information after module load

[PatTheMav]

Description

Splits the initialisation of encoder information for a loaded profile from the profile activation as encoder information is not available at the time of activation during program initialisation.

Motivation and Context

The function InitBasicConfigDefaults2's purpose is to set up encoder-related settings for a profile, which requires all available encoders to have been discovered first.

As encoders are added by runtime modules, this also means that discovery needs to be delayed until after modules have been loaded, but modules themselves require a valid profile to exist and be loaded.

This leads to the requirement of profiles needing to exist in a half-initialized state when modules are loaded and only becoming fully initialized after that.

This distinction is not necessary after the initial launch of obs-studio because runtime plugins are not hot-loadable, so the update can happen unconditionally at any time a plugin is changed, which implicitly couples every call to ActivateProfile with a call to UpdateProfileEncoders.

Fixes #11464.

How Has This Been Tested?

Tested with profiles using hardware H264 and HEVC encoders and checked that their selection was retained after closing and opening OBS as well as switching between different profiles that had any encoder except for X264 configured and checked that those profiles' encoder settings were also kept.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[DeeDeeG]

Hello, thank you for working on this issue. I have tested this, and I have mixed results to report.

First half of results: The originally reported steps to reproduce are fixed by this change. 👍

Testing details and notes (click to expand if you want):

Before this change, CheckForSimpleModeX264Fallback() is called twice during startup before any UI is shown to the user. The first time being before any encoders are actually loaded internally, so the checks fail and fallback to x264 happens 100% of the time for Simple mode streaming and recording codec, which is undesired.

WITH this change, CheckForSimpleModeX264Fallback() is only called once during startup, and it is after the encoders are loaded, so the checks work correctly, and unexpected fallback to x264 is avoided. 👍

Checked with Visual Studio on Windows 10 for debugging purposes, and confirmed the fix with the PR's CI-generated binary as well.

---

Second half of results: Unfortunately, there is now a crash ❌ upon making a new profile (menu, "Profile" -> "New" in the UI, enter a name and press "OK"), as get_simple_output_encoder() is called with encoder being NULL, and there is an "access error".

Call stack of get_simple_output_encoder() when the access error occurs (when making a new Profile) (click to expand):

ucrtbased.dll!00007ffe207560c0()
obs64.exe!get_simple_output_encoder(const char * encoder) Line 549
	at C:\Users\User\obs-studio\UI\window-basic-main-outputs.cpp(549)
obs64.exe!SimpleOutput::SimpleOutput(OBSBasic * main_) Line 647
	at C:\Users\User\obs-studio\UI\window-basic-main-outputs.cpp(647)
obs64.exe!CreateSimpleOutputHandler(OBSBasic * main) Line 2531
	at C:\Users\User\obs-studio\UI\window-basic-main-outputs.cpp(2531)
obs64.exe!OBSBasic::ResetOutputs() Line 1999
	at C:\Users\User\obs-studio\UI\window-basic-main.cpp(1999)
obs64.exe!OBSBasic::ResetProfileData() Line 748
	at C:\Users\User\obs-studio\UI\window-basic-main-profiles.cpp(748)
obs64.exe!OBSBasic::ActivateProfile(const OBSProfile & profile, bool reset) Line 709
	at C:\Users\User\obs-studio\UI\window-basic-main-profiles.cpp(709)
obs64.exe!OBSBasic::SetupNewProfile(const std::string & profileName, bool useWizard) Line 79
	at C:\Users\User\obs-studio\UI\window-basic-main-profiles.cpp(79)
obs64.exe!OBSBasic::on_actionNewProfile_triggered() Line 445
	at C:\Users\User\obs-studio\UI\window-basic-main-profiles.cpp(445)
obs64.exe!OBSBasic::qt_static_metacall(QObject * _o, QMetaObject::Call _c, int _id, void * * _a) Line 3435
	at C:\Users\User\obs-studio\build_x64\UI\obs-studio_autogen\include_Debug\EWIEGA46WW\moc_window-basic-main.cpp(3435)
obs64.exe!OBSBasic::qt_metacall(QMetaObject::Call _c, int _id, void * * _a) Line 3725
	at C:\Users\User\obs-studio\build_x64\UI\obs-studio_autogen\include_Debug\EWIEGA46WW\moc_window-basic-main.cpp(3725)
[External Code]
obs64.exe!OBSApp::notify(QObject * receiver, QEvent * e) Line 1438
	at C:\Users\User\obs-studio\UI\obs-app.cpp(1438)
[External Code]
obs64.exe!OBSApp::notify(QObject * receiver, QEvent * e) Line 1438
	at C:\Users\User\obs-studio\UI\obs-app.cpp(1438)
[External Code]
obs64.exe!run_program(std::basic_fstream<char,std::char_traits<char>> & logFile, int argc, char * * argv) Line 2102
	at C:\Users\User\obs-studio\UI\obs-app.cpp(2102)
obs64.exe!main(int argc, char * * argv) Line 2639
	at C:\Users\User\obs-studio\UI\obs-app.cpp(2639)
obs64.exe!qtEntryPoint() Line 50
	at D:\a\obs-deps\obs-deps\windows_build_temp\qt6\qtbase\src\entrypoint\qtentrypoint_win.cpp(50)
obs64.exe!WinMain(HINSTANCE__ * __formal, HINSTANCE__ * __formal, char * __formal, int __formal) Line 60
	at D:\a\obs-deps\obs-deps\windows_build_temp\qt6\qtbase\src\entrypoint\qtentrypoint_win.cpp(60)
[External Code]

---

[PatTheMav]

Hello, thank you for working on this issue. I have tested this, and I have mixed results to report.

First half of results: The originally reported steps to reproduce are fixed by this change. 👍

Testing details and notes (click to expand if you want):
Second half of results: Unfortunately, there is now a crash ❌ upon making a new profile (menu, "Profile" -> "New" in the UI, enter a name and press "OK"), as get_simple_output_encoder() is called with encoder being NULL, and there is an "access error".

Call stack of get_simple_output_encoder() when the access error occurs (when making a new Profile) (click to expand):

Pushed an update, please check the use cases again once CI has finished creating the artefacts.

[DeeDeeG]

Okay, with the latest change, all looks good to me. 👍

• Launching OBS does not reset a usable hardware encoder such as nvenc or nvenc_hevc to x264 unexpectedly for Simple mode
• Making a new profile from the menu ("Profile" -> "New") does not cause any crash
• Fallback still works from an unavailable encoder (such as qsv_av1 on my machine) to x264 where needed/expected to do so

I confirmed the above with this branch checked out in debug mode in Visual Studio on Windows 10, as well as with the PR's CI binary (also on Windows 10, same machine).

(Flow of code execution stepping through in a debugger seems reasonable to me also, though I'm not super familiar with the code admittedly. Nothing stood out as being off, end-user-apparent behavior seemed pretty much the same as 30.2.3 as far as I could tell.)

[RytoEX]

Looks okay at a glance.