Local development QoL for private repositories

[matthewelwell]

Current problems

• Cross module dependencies are painful to work with, e.g. flagsmith-workflow -> flagsmith-common and flagsmith -> flagsmith-common causes conflicts when trying to develop on the flagsmith-workflows repository
• Running integration tests is painful from private repositories

TODO

• Create our own private package repository and deploy all flagsmith modules there so that we can use semver properly (so we can implement version ranges)
  • AWS CodeArtifact
    • Rough estimate of cost worked out at ~$165 per year.
• Write publishing workflows for each of the packages, including for pull requests

Suggested improvements

• Move all integration tests out of private modules, into core repository (or a separate repository if we have concerns about the privacy of this code)

[matthewelwell]

We originally created a CodeArtifact repository (currently in eu-west-2) to host public packages (e.g. flagsmith-common), however, it appears that it is not possible to make CodeArtifact repositories open to the public without additional infrastructure. As such, we plan to only use CodeArtifact for private modules (e.g. flagsmith-workflows, flagsmith-saml, etc) and we will just publish the public modules directly to pypi.

We have started a PR against the workflows repository here.

Next steps:

• Create an IAM user (role?) specifically for publishing CodeArtifact

[matthewelwell]

As mentioned above, and discussed in a recent engineering meeting, it seems as though hosting the public packages in pypi (e.g. flagsmith-common) is still the best way forwards.