The future of Mozart - we're at a crossroad!

[coenjacobs]

Hello everyone! 👋 Now that Mozart is being used by more projects, more people are getting involved and, of course, more issues have come up. Some things haven't been properly thought out by myself, have become some kind of technical debt already, but others have been unforeseen. This is exactly why I've left Mozart in a sort of beta state for quite some time (mind you, the first release was tagged in March of 2017), to give myself some time to properly understand how people would use it.

Mozart has become quite stable, over the past couple years. Among others, WooCommerce has started using Mozart, WP Rocket has implemented it and the people of Nextcloud have even found usages for Mozart outside of the WordPress ecosystem. There are days where I think the current master branch is ready to become the first proper stable release: 1.0.0. Then there are other days, where I feel the 1.0.0 release is years away. We simply haven't figured out the full picture, yet.

What we have figured out with Mozart, is that we're at a crossroad. Mozart has organically grown to the place where it is right now. There are two major issues with Mozart, which are rather hard to solve considering the current setup. In this document (no worries, it's long, but I've provided a TL;DR) I'll explain why this is so hard to solve.

What the decision is about (also known as, TL;DR)

One of the issues has to do with the build flow of a piece of code, after it is being transformed by Mozart. The other, is that since Mozart is designed to be a dependency, it influences the packages being installed. For example, Mozart requires PHP 7.2 currently, but what if your project only requires 7.0 or even worse 5.6 still? You can't realistically install Mozart in a package that also wants to maintain compatibility with PHP 5.6.

Issues (among others) that made me think about this decision going forward:

• How to run Mozart on a composer.json that has platform requirements locked to an older PHP version? How to run Mozart on a composer.json that has platform requirements locked to an older PHP version? #95
• Conflict of installation as "require-dev" and "composer i --no-dev" Conflict of installation as "require-dev" and "composer i --no-dev" #79

For me, there are basically two scenarios in order to solve this:

1. Keep Mozart as a part of the project (install it as a development dependency) and work around the known issues. Not ideal, in my humble opinion, as this will involve even more monkey patching as what Mozart is currently doing, but it does stay true to the original design and philosophy of Mozart: easy to install, low-impact and low effort to maintain.
2. Change Mozart into a tool that is not tied to the project, that can be installed as a developers tool (for example as a global Composer package). This would make the two main issues significantly easier to solve (as Mozart is no longer part of the project, apart from the configuration), but also makes it a tad more complicated to setup and maintain. The development team for example needs to keep their install of Mozart up to date and probably on roughly the same version.

That's the main summary of what the rest of this lengthy issue is going to be about. Let's dive in the details.

Background and philosophy of Mozart

I'd like to give a little context on what pushed me to make Mozart in the first place.

Mozart is designed to bridge the gap between the WordPress ecosytem and the vast packages ecosystem of PHP as a whole. Since WordPress is such an end-user friendly focussed CMS (for good reasons), there is no place within the ecosystem where an end-user would be burdened with using a developers tool like Composer. Also, since WordPress has to run on basically any hosting infrastructure, running Composer to install packages from the administration panel (trust me, I've tried - it can be done) is a mission impossible to make it happen and compatible with every server out there.

But why make a new tool for this? There are other tools that enable you to do this, right? Yes, there are now. PHP-Scoper, for example. PHP-Scoper is a fantastic tool, that does the job right. But, PHP-Scoper wasn't available back when I started the Mozart project. Also, PHP-Scoper has a few limitations (no support for classmap autoloaders, for example) that were and are still quite common within the WordPress ecosystem. Finally, PHP-Scoper can be quite the tool to add to your development flow, while Mozart was designed to be simple to implement, specifically tailored for WordPress projects.

The key values of what's important to Mozart:

• Must be able to be easily installable by a developer, preferably in a specific version.
• Distribution must be done through an existing package manager or easily maintained separately.
• Shouldn't add a whole layer of complexity to the development process, i.e. learning a whole new tool/language.

Mozart always has been and always will be geared towards solving the conflicting dependencies problem in the WordPress ecosystem, as efficiently and opinionated as possible. By being opinionated in certain ways and specifically focussed on WordPress projects, Mozart has quickly become easy to understand and implement. These key values have resulted in what Mozart is today, but they have also resulted in the issues the project is currently facing.

Solution 1: Keep Mozart as a development dependency

In terms of ease of implementation and maintenance, this is the highly preferred route. It makes certain issues harder to deal with, if not near impossible. We'd have to tackle the issues in a couple hacky ways, working around the issues that are listed above. I can think of a few ways to deal with the two issues I've mentioned above, but they aren't pretty. This is where ease of use, takes away from being a package that can provide your project with the best end result.

Where Mozart as a development dependency is easy to implement and maintain, it becomes a lot harder to deal with when it comes to releasing the code. The current approach makes it harder to run through CI (because it actively promotes you to setup Mozart to run in a Composer script after installing packages) and harder to use when you simply want to make a zip file of your project and distribute it from there. Finally, the fact that Mozart itself introduces a few dependencies to your project (which are only development dependencies, but still added) can be a burden as well.

Solution 2: Change Mozart to become a standalone package

The downsides of the first solution are almost immediately solved by this solution. This would basically come down to having similar installation steps as when installing PHP-Scoper via Composer, installing it as a global package. This would add some complexity for the implementing development team, as each developer would have to install Mozart on their machine by themselves, in roughly the same version.

The configuration for Mozart can still be done in the implementing projects composer.json file though, as it currently does. Maybe that way, we can have some sort of compatibility check for the Mozart binary to be in the right version? There are a lot of unknowns to me in this situation and that's why I'm writing this post. I need some help figuring out the best approach.

Yes, this solution would make Mozart very similar to PHP-Scoper. Yet, the underlying philosophy and Mozart being specifically designed to work in and for the WordPress ecosytem will help maintain it's value. Even if Mozart were to become a standalone package, working outside of the projects it is implemented in, it should still be far easier to implement as compared to PHP-Scoper for your WordPress projects.

The future of Mozart

The past two-and-a-half years have been a fun ride. Mozart has become a usable (and often even useful 😉 ) package, where the first release was a total hack job. I'm trying to spend as much time on the project as I humanly can, but it is still an unpaid side project (it feels obligatory to mention here, you can sponsor me on GitHub to help me find more time for the project).

If you've read all the way down here - thank you! I feel that this is an important decision going foward and not one that I'm taking lightly. Mozart was born out of necessity and has since been embraced by lots of developers. Thank you all for contributing to this project, I know you all do this in your own free time and I am forever grateful that you're taking the time to help my humble project forward. I'm very much looking forward to hearing your ideas.

[XedinUnknown]

I have tried using PHP Scoper recently. The way they solve the dependency problem is:

1. PHP Scoper itself gets scoped.
2. The scoped code is then packaged into a Phar.
3. Phar can easily be installed with Phive.
4. PHP Scoper thus does not factor into the dependency graph of your project.

I believe that since Mozard, PHPScoper, and everything similar are monkey-patching tools, they should be used during build, i.e. developers should not be working on patched code. I also believe that it is very important to preserve interoperability in the subject PHP code, and for this it is necessary to be able to exclude FQNs/namespaces from being scoped. So, IMHO the approach with making Mozart stand-alone is the one that should be used, because it supports the correct workflow. Sadly, without the ability to exclude namespaces, none of such tools are viable for me.

[Luc45]

As WordPress matures and bumps the minimum PHP version, dependency conflcits will get ever more common, and will not go away.

Realistically, centralized dependency management for plugins is something that I don't see happening in Core any time soon, and I don't even think it should, as it could prevent you from installing certain plugins for having incompatible dependencies.

Scoping dependencies seems the way to go, and having a tool that can do that specifically on the WordPress environment is very important. WordPress core should consider bumping resources into this effort.

PHP-Scoper is a great tool, well built, and tested by competent people. However, they are not interested nor focused on the WordPress environment, and their Composer autoloading support is almost inexistent, forcing you to come up with a complex setup to autoload your prefixed dependencies alongside your unprefixed source code.

I personally use PHP-Scoper in WP STAGING, implementation details can be found here.

I'm a profound admirer of the philosophy Simplicity is the ultimate sophistication, and I think that anything that has a very high technical barrier will never be popular in WordPress, which is a double-edged sword. It's nice because it's who we are, but it also can fall down into being simplistic.

I agree with @XedinUnknown that Mozart should probably be a standalone tool, not bound to the dependency constraints of a given project. As for scoping being done as part of a build process, that is the case for the current scenario of using Composer class maps, but if it could work dynamically without class maps for development, that would be nice, too.

[XedinUnknown]

From my experience, no autoloading is strictly necessary to perform this sort of patching. Me and @mecha are working on a prototype that does exactly that, without constraints of autoloading, packages, files, etc. It works only on the AST, which means that the formatting will not be preserved. Given that we're talking about build-time monkey-patching, however, I don't feel that code formatting in the eventual product is really a concern, as long as that code works in exactly the same way. That said, the code will be formatted closely to the original, but not guaranteed.

Here's a little post I had written about package management in WordPress. I describe problems and potential solutions, if anyone's interested.

With regard to PHP Scoper's support for WordPress, it isn't entirely accurate that they are not interested in this. I see that @Luc45 has already referenced that issue somewhere, but for book keeping it's humbug/php-scoper#303.

[Luc45]

Regarding the article, I can't see a way to go about this other than prefixing the dependencies.

Even if we manage to implement something in Core that would allow different versions of the same package to be installed (like NPM does), the PSR-4 standard (simply put, namespaces), does not allow to specify different versions of a package to use.

Centralized dependency management would require constraint compatibility between all plugins. I find it hard that WordPress would delegate this limitation to the users.

In my opinion, it's more feasible to have a very solid, tested, reliable, and easy to use tool for Plugin developers to prefix their dependencies.

no autoloading is strictly necessary to perform this sort of patching

I'm not sure if I follow you. Can you please elaborate? I've just prefixed WP STAGING using PHP-Scoper, loading the classes with an autoloader.

[jenkoian]

This issue got me thinking a little bit. I hadn't really come across Mozart before (but have indeed suffered from the problems it tries to resolve).

Anyway, it got me thinking that this is nice but it requires buy in from the plugins, and some (many?) don't seem to care (or aren't aware) of the problem so much.

So I was wondering if there was a way to perhaps 'warn' the user or something that they could be in for some trouble when using a plugin (in the same way composer will just fail if you try and install a package with incompatible dependencies). Often these problems are quite hidden until you notice something not quite right, sometimes on a live site! (this post does a good job of describing that https://wptavern.com/a-narrative-of-using-composer-in-a-wordpress-plugin).

So I made a start on a very rough and ready composer plugin https://github.com/jenkoian/wp-plugin-trouble-detector, (I just realised this won't do a good job for packages form wpackagist, so maybe I need to look at that) but anyway don't want to derail this conversation too much, but am interested in the problem space and if anyone's up for discussions on how to raise awareness/improve the 'narrative' for the end devs, I'm in!

[jenkoian]

Thinking again from the end user (dev) perspective, knowing absolutely nothing about how mozart (or similar tools) actually work, could you run it as a composer plugin (I mean it is already a composer plugin but I mean as the end user (dev))? So the emphasis moves from the plugins to the end user (dev)? I guess you'd want to offer both? So maybe a mode where it attempts to 'force' a plugin to using mozart? Is that even viable? Am I even making sense?

[mecha]

The concern for code patching should, IMO, fall onto the author of the software that is being patched; i.e. the WordPress plugin developer.

Addressing this concern from a site-owner standpoint implies the existence of a solution that is 3rd party to the WP plugin that a developer would produce. For example, a standalone Mozart WordPress plugin, or a Composer plugin that does what @jenkoian proposed. The implication is that WP plugin developers would no longer need to concern themselves with patching their code; great! But that leaves the developer in a weird position of stasis, where they're not actively doing anything to mitigate plugin conflicts and leaving everything at the mercy of site owners' diligence and abilities.

I don't think that's a situation that a sensible plugin developer would be comfortable with. Speaking from experience, asking site owners to change their WordPress environment (for instance, disabling WP Cron and using a real cron), even if beneficial to more than just your own plugin, is very hit or miss. Most won't bother and will move on to a different plugin. The stakes are higher for plugin developers.

As such, I agree with @XedinUnknown and @Luc45. The best solution is probably one that plugin developers use during some phase in their build process, producing a conflict-free plugin. In other words, a standalone tool.

[jenkoian]

The implication is that WP plugin developers would no longer need to concern themselves with patching their code; great! But that leaves the developer in a weird position of stasis, where they're not actively doing anything to mitigate plugin conflicts and leaving everything at the mercy of site owners' diligence and abilities.

Yeah, fair, I don't disagree it should be on the plugin devs shoulders, I'm just trying to think of the (non hypothetical) case where the plugin developers won't bother and thus leaving the site-owner (btw much nicer term than end-user that I was using 👍 ) with little option.

[jenkoian]

If anyone interested I updated my little composer plugin https://github.com/jenkoian/wp-plugin-trouble-detector to support wpackagist and made it actually work (😆 ) and stuff