Rebuild Koji in new infrastructure and with new architecture

[ehelms]

Discourse discussion: https://community.theforeman.org/t/rebuilding-koji/20977

Notes:

• Current Koji has server, builder, database all in one machine
• Requires a separate builder to handle EL8 running Fedora
• Is unmanaged by any config management

Server and hostnames layout for new Koji:

• Koji server
  • Hub + database
  • hostname: koji01.aws.theforeman.org
  • service name: koji.theforeman.org
• Koji builders
  • hostname: builder0X.koji.aws.theforeman.org

Additional improvements to consider:

• Reduce the amount of swap being used comment
• Systemd dependency on mount points comment
• Drop use of /etc/hosts due to floating IP comment
• Get rid of random hostname comment
• Enable SELinux comment
• Add /usr/local/bin/foreman-root-restrict.sh to source control

Action Items:

• Build out Ansible or Puppet to setup and manage Koji infrastructure
• Build a new environment with config management on new infrastructure
• Manage Koji within Foreman instance
• Migrate to new disk format

[ekohl]

Right now the httpd config also breaks on Koji upgrades. That should also be fixed. Don't have a more precise note, but it was noticed during the last upgrade.

[ekohl]

On hold due to #1795.