always execute access_permissions_test from Foreman core

[evgeni]

No description provided.

[ekohl]

I wanted to avoid this and looked at patching plugins. Now that I look further I see a common pattern in some plugins to include from core.

https://github.com/theforeman/foreman_puppet/blob/master/test/unit/foreman_puppet/access_permissions_test.rb
https://github.com/theforeman/foreman_statistics/blob/master/test/unit/foreman_statistics/access_permissions_test.rb
https://github.com/theforeman/foreman_rh_cloud/blob/foreman_3_9/test/unit/rh_cloud_permissions_test.rb
https://github.com/theforeman/foreman_bootdisk/blob/master/test/unit/access_permissions_test.rb

Then foreman_openscap has this surprising bit:

https://github.com/theforeman/foreman_openscap/blob/09d1197ee823cfa7d1425f6498ba0686dcbf8a12/lib/tasks/foreman_openscap_tasks.rake#L95-L107

I'm still leaning to making sure rake test:$plugin does the right thing, because that's what developers should also run locally.

[evgeni]

So, we could have a test:things_plugins_need_to_test_from_core_and_naming_is_hard that lists access_permissions and test:$plugin depends on that (essentially, what openscap is doing plus an dependency)?

the other linked things are good own tests (as in: they are testing that the routes done by the plugin are sane), but I think executing the core version of the test also ensure we don't break stuff from core?

[ekohl]

So I found how I tried to solve it in the past: theforeman/foreman-tasks@3ad347b can be summed up as theforeman/foreman_plugin_template#51

[kamils-iRonin]

I like the idea of having a static list of recommended core tests in the core. I would also run all core tests on merge to master – to make sure that a plugin does not affect the core. Of course, making these core tests optional would still give us some flexibility.