Preview - Keycloak Test Framework

Signed-off-by: Lukas Hanusovsky <[email protected]>

blog/2024/preview-keycloak-test-framework.adoc

@@ -0,0 +1,42 @@
+:title: "Writing new tests has never been easier than before." Keycloak Test Framework
+:date: 2024-08-28
+:publish: true
+:author: Lukas Hanusovsky
+
+== How It All Started
+
+The idea to replace the current https://github.com/keycloak/keycloak/tree/main/testsuite[test suite] has been on the table for multiple years. Initially, it was meant to be only a refactoring of the current approach on how to write tests, but after a few internal discussions and refactor updates it turned out a new test suite, based on a https://github.com/keycloak/keycloak/tree/main/test-framework[new framework] would be a better solution.
+
+It would be good to mention a few drawbacks, that stand out when working with the current https://github.com/keycloak/keycloak/tree/main/testsuite[test suite]. First of all, is the complexity of various configurations and additions made on top of the https://arquillian.org/[_Arquillian framework_]. These changes make the test suite powerful, but the cons is that without proper documentation for beginners is almost unreadable. The second thing has the same importance, the _Arquillian framework_ is not fully supported anymore. Other things to mention are a complicated execution system, where you want to specify what exactly should be tested, then abstract classes with shared configurations, missing the option to add a custom extension, and last but not least, the massive amount of tests without knowledge if they are needed anymore.
+
+== Brighter Future?
+
+Keycloak team had started a https://github.com/orgs/keycloak/projects/42[joint effort] between the engineering and quality engineering people to design a new test framework (May 2024). It had been initiated with a prototype to verify if our thoughts were feasible. The prototype was based on the _JUnit5_ testing framework, specifically to implement _JUnit5_ callback classes which extend the default test lifecycle functionality and provide custom inject annotations, like _@KeycloakIntegrationTest_, _@InjectWebDriver_ or _@InjectRealm_.
+
+For our purpose, we have https://github.com/keycloak/keycloak/blob/main/test-framework/src/main/java/org/keycloak/test/framework/KeycloakIntegrationTestExtension.java[implemented] these three callback classes:
+----
+BeforeEachCallback
+AfterEachCallback
+AfterAllCallback
+----
+
+After a successful test round, we've continued with a proof of concept extending features list to support multiple server modes, different databases and _WebDrivers_, clients and users setup, _SmallRye_ configuration support, _OAuthClient_ based on https://connect2id.com/products/nimbus-oauth-openid-connect-sdk[_Nimbus SDK_] (this feature is a preview only) etc. The full list of currently implemented features can be found here in the https://github.com/keycloak/keycloak/tree/main/test-framework/src/main/java/org/keycloak/test/framework/annotations[annotations package].
+
+== Are you curious about where to start?
+We suggest reading the https://github.com/keycloak/keycloak/blob/main/test-framework/README.md[user guide], which will provide a basic overview of how the framework works and should be used. If this is not enough, you can also check https://github.com/keycloak/keycloak/tree/main/test-poc/examples[examples module].
+
+For extension developers we recommend to look into the showcase example on how to start _Keycloak_ with their custom provider - link.
+
+Once you find a bug, want to discuss something, or propose a new enhancement, please follow this GitHub feedback https://www.github.com/discussion#blabla[discussion] link.
+
+== Coming Soon
+
+Preview - https://github.com/keycloak/keycloak/tree/main/test-framework[_Keycloak Test Framework_] will be introduced in _Keycloak Nightly_ builds. We plan to add more features, which will cover 80% of functionalities currently available in the https://github.com/keycloak/keycloak/tree/main/testsuite[integration test suite]. We also plan to introduce some new concepts, e.g. _@KeycloakTest_ annotation, which will add an option to test providers without the _Keycloak_ server running. It should be based on the _KeycloakSession_ class. Other interesting features should be parallel execution on a GitHub runner, adding a custom extension to a Keycloak server, or enabling a Keycloak cluster setup.
+
+== Acknowledgement
+
+I would like to thank all the people who put the proof of concept together and made it real: Miquel, Simon, Filip, Moises, Jon, and Pedro. A special thank you goes to Stian, who led the technical design and proposed very nifty things that raised the project to another level.
+
+Thank you for your feedback.
+
+Enjoy!