Backward compatibility - what it is and when it matters

[rivantsov]

Introduction - why discuss such a basic thing?

One of the established rules for GraphQL spec development is "preserve backwards compatibility", and nobody argues against that.
It might seem that the term does not need any explanation - everybody knows what it means. However, based on few occasions here in GraphQL WG, in discussions and posts, it looks like different people have different interpretations of what it means and when it applies. I think sometimes we do not apply this rule correctly and misjudge the suggested change - people claim it breaks the 'backward compatibility', while it does not.

As an example, I was told allowing recursive fragments would break backward compatibility. "We remove the restriction no-recursion and it is backward incompatible". - Really? I am not sure. I will bring another example from our meeting discussions down below.

The GraphQL spec itself I think uses the term incorrectly in one place; more on this later.

In expectation of coming discussions of new features (recursive fragments, using input types as outputs, others), and expecting backwards compatibility considerations raised in these discussions, I believe we need to discuss this subject separately and upfront, to avoid arguing about this in the midst of other topics.

I suggest MY understanding of the term in this post, and how it should be applied in WG work Please comment and give your opinions. But I do believe that we need some SHARED understanding of the subject and how and where it applies.

Backward compatibility - software in general.

Old app code continues to work on newer versions of the underlying infrastructure and dependencies

This always presumes some limited forward compatibility on behalf of the app itself - a tolerance to some minor unexpected extra things. Don't break if you see some extra, just ignore it. And you cannot do anything about an app that just refuses to work with anything but a specific version (that was the hell with WS and SOAP, exact namespaces and specific versions of any single type, remember that?)

GraphQL backward compatibility - "narrow" definition

Backwards compatible spec change from versions V1 to V2 means that any GraphQL document(schema or query) valid for V1 is valid for V2.

Therefore:

1. Allowing in V2 things that were not allowed in V1 is backwards compatible. So allowing recursive Fragments in V2 is backward compatible change from V1 when they were not allowed. For a simple reason: old queries compliant with V1 never used recursive fragments, they were not allowed, so they should continue to work.
2. Allowing directives in new places is backwards compatible, for the same reason

Runtime and modifying standard structures (Introspection types, Error type)

Moving server infrastructure from V1 to V2 of the spec can cause some automatic changes in behavior, even for old V1 queries. Server responses might change. Example: introspection queries, new specifiedByUrl field. Another example, Error object: we might add a 'code' field (as had been suggested before), so old clients and apps might suddenly start seeing a new unknown field in errors with probably some default value.

In this case, GraphQL (WG) presumes the above mentioned limited forward compatibility of implementations - they should not error out if they encounter something extra in standard types.

Target areas of concern - apps, libraries, tools

1. GraphQL apps - (end apps) - main concern, they need backward compatibility of their GraphQL assets (queries, schemas). For many apps, there might be no dev team behind it, no more, to make adjustments. Project is done, pushed to prod, team of consultants is disbanded.
2. GraphQL frameworks, client and server - less concern, devs of these libraries have advanced warning about coming changes (spec draft), and they have time to upgrade. Our main goal on spec side is to make sure it is reasonably doable: it is possible to build a new version of framework for up version of spec that is backward compatible for old features, all old stuff works the same.
3. Tools (Graphiql) - even less concern. Tools do not run in production. Just like framework/library makers, tool makers also have advance warning about coming changes in draft spec, so they have time to build new version and release it even ahead of time - and this upgrade should be backward compatible and work with still current version of the spec.

Regarding tools, an interesting case from the past. On one of the meetings we were discussing some addition to introspection, I think it was "appliedDirectives", but I'm not sure. One of the objections was "Look, if we add this, but user uses some older version of a tool to inspect schema, the tool won't show this new field, so user will be unaware of some information, and this is really bad, so this change is not backwards compatible". And this kind of closed the discussion. Needless to say, I strongly disagreed. But starting arguing about 'what backward compatibility really is' was out of question, out of scope for the meeting, so this argument "won".
And this is why I am bringing it up here, as a separate discussion, to hopefully settle the issue for the future.

Now about the issue with tool itself. First, the rule is - use up-to-date tools, period. At least matching the version of API or server you are looking at. It is NOT WG's concern that somebody uses old tool. Secondly, the hypothetical case presented is NOT about backward compatibility, it is more about lack of reasonable 'forward compatibility of the tool". When encountering some unexpected element in the introspection type, the tool should not have ignored it, but should try to show the info to the user in some way, at least alert him/her that there's more. This is more like a fault on tool developers.

And third, following this logic, it is impossible to do any progress in the spec at all, because for any change there might be some old tool installed somewhere that does not know about this new thing.

Summary: suggested basic principles for advancing the spec

1. Capabilities: do not remove, add only. Do not change, extend only
2. Restrictions on existing elements: never add one, but removing restriction should be OK.
3. Presumed some tolerance of implementations to unexpected extra elements - do we mention it in the spec?
4. Primary concern with existing end apps and their GraphQL schemas and queries;
5. Much less concern with libraries and tools, just publish spec draft early enough.

Improper use of the term backwards compatibility in the spec

Section Deprecation tells us that @deprecated directive is "to support the management of backwards compatibility...".

I think this is incorrect, deprecated is not about compatibility at all! It is about coming INCOMPATIBILITY. With backward compatibility the old code continues to work, you do NOT need to change/adjust it. In contrast, Deprecated means that old code will STOP working, an early warning that you'd better change your code soon, or it will start failing.

I suggest to change this paragraph to something like:

@deprecated directive helps to manage the gradual evolution of application schema, giving an advanced warning to clients about upcoming changes in the schema.

Being pedantic: Backward or Backwards?

Interesting explanation here: https://www.quickanddirtytips.com/articles/backward-versus-backwards/

So it looks like it should be always "Backward compatibility" without 's' for an adjective, and either with or without it for an adverb, both forms are OK: "Backward compatible" or "Backwards compatible"

Real life case: Inputs as Outputs implementation and encountered challenges

Some time ago I was involved in a GraphQL service project that had many simple types, data containers with a few properties. These objects were submitted to server by a producer service, and then queried by other multiple clients. So the types were used as both input and output types. That by the way is a very common case I believe.

I started with defining identical input and output types, lots of them. The schema bloated, and it just did not make sense. So I decided to just implement in NGraphQL this long requested and discussed feature - allowing returning input types as field output types. The basic idea for implementation: "Input types are just restricted Object types, so any Input type is also Object type." Restrictions are - no arguments on fields, and only primitive or Input types for fields. So there are no separate collections of Input and Object types, just Object types, and some have IsInputType flag set. But introspection returns them in separate collections, as before. They are still declared as Input and Object types in the schema and everywhere.

So I did implement it, and here are the findings; they may be interesting in the context of this discussion

1. The change is backward compatible, as for GraphQL language and protocol. Old schemas never used Input types as Outputs, so the old code will never encounter this feature, and thus never fail.
2. Implementation on the server side was more challenging than I expected, honestly. It required a bit of restructuring of the code, merging internal Input and Object types representations and their capabilities, etc., and then cleaning up the mess and retesting everything. Not as primitive as declaring "input type is just restricted output type". But it's OK, I got it working.
3. Graphiql introspection query failed, with error something like "expected Object here", when it encountered Object field of type "Input type". Graphiql continued to work, I could run queries, but model tabs were empty.

My statement: this change is backwards compatible, and troubles with Graphiql are NOT a break of backward compatibility. The tool(s) can be fixed if this feature is put into the spec.
Change my mind :)

[benjie]

It feels to me that you're arguing against a misinterpretation of what the WG consensus is, so discussion definitely feels warranted to remove any ambiguities in communication. Allow me to address just a few of your initial points with my understanding:

if we add this, but user uses some older version of a tool to inspect schema, the tool won't show this new field, so user will be unaware of some information, and this is really bad, so this change is not backwards compatible

I highly doubt that there was any WG consensus behind this statement, at least as currently stated. @specifiedBy is one example where extra information is available, but because old GraphiQL does not query for that data it does not receive that data and thus cannot be confused by it. Old tools not reflecting new features is absolutely fine, and part of the backwards/forwards compatibility of GraphQL. This is why we require selection sets and don't allow wildcards! Another example of this would be the @oneOf directive, though it hasn't been merged yet - in this case an old client would just see it as a regular input object and not know about this special behaviour, but that's deemed acceptable - it should not break anything, just may require care when users deal with it.

And third, following this logic, it is impossible to do any progress in the spec at all, because for any change there might be some old tool installed somewhere that does not know about this new thing.

The GraphQL spec is advancing, so I guess this is one of the contradictions in your logic that has helped you notice your understanding of the WG's consensus must be flawed, hence seeking this clarification. To start addressing this, I suggest that you carefully read the guiding principles themselves - they're what a lot of the GraphQL specification discussions centre around, directly or indirectly - and further note that they are guides rather than rules. In general: the benefit of any new feature must outweigh the cost.

To address the quote directly: adding information to the introspection schema should generally not break any old tools because old tools only query fields that they knew about at the time, and we add new information via new fields so as to not break old tools. This is one of the reasons why the struct RFC maybe should enhance the Scalar type rather than the Input Object type - existing tools that see a Scalar will, in the vast majority of cases, not even look at the "fields" result, and thus adding information there should not confuse/break existing clients.

I was told allowing recursive fragments would break backward compatibility. "We remove the restriction no-recursion and it is backward incompatible"

It isn't backwards incompatible on existing operations - existing operations are unaffected since they do not use recursive fragments (as you note). It's backwards incompatible on GraphQL schemas, because if the GraphQL library that powers a GraphQL schema is updated to have this restriction removed, suddenly the schema can receive queries that it was not expecting and go into an infinite loop. Previously the schema was relying on the "no recursive fragments" rule to ensure that it was safe, removing this rule would be a backwards incompatible change for the schema, and could introduce a serious DoS attack vector.

This may be the case for your "GraphQL apps - (end apps) [...] Project is done, pushed to prod, team of consultants is disbanded" example, where a maintenance crew comes in and updates all the dependencies.

Because of this risk to existing schemas, this is a significant consideration, and if we were to make this change we'd have to signpost it very heavily to ensure that as few people as possible were negatively impacted by it.

[rivantsov]

thanks for the feedback!

... you're arguing against a misinterpretation of what the WG consensus is, ...

no, I did not say anything about consensus, the problem is that I do not know what consensus is, on a more detailed level that I try to discuss here. What I say is that sometimes an argument "this breaks backward compatibility (BC)" pops up, and it is kind of so powerful that it works as a killer argument. Not as consensus of WG, but as an argument in particular case; wrong argument in some cases I think, but since there's no time/place to discuss BC in the middle of other topic, it kinda works as an objection.

As a particular example, you argue that recursive fragments break BC, 'on schemas' as you say. I argue it is not. Endless loops - yes, it is a problem. But it is NOT BC. Because after statement 'all old stuff works', BC consideration is gone. Endless loops are a problem, but of different nature.
Let's discuss it.

[yaacovCR]

As far as I recall the discussion with regards to recursive fragments, the conclusion was that any execution engine would presumably include its own depth limit, and that it would probably pay to formalize the depth limit as a directive on the fragment.

What that proposal was waiting for was a champion to advance an implementation and then the remaining question was whether this increased complexity would be worth it considering u could always add this as a “macro” where operations with recursive fragments with a depth limit were transformed into a non recursive version.

For what it’s worth, I think that this feature is worth adopting, I just didn’t have time or really so much personal interest in championing it!

I am not weighing in on your larger point — sort of on purpose. The point of this comment is just to clarify the state of recursive fragments.

[rivantsov]

About recursive fragments. I think it is coming back, I believe some other user tried to put it on future agenda. @benjie actually acknowledged that it is a useful feature (allow recursion) in his 'struct' proposal, but only in case of structs, and it's still not clear to me how it supposed to work there, but not with regular fragments

Again, this is just an example for BComp issue

[mjmahone]

One thing to note: we attempt to preserve compatibility in both directions. Existing documents should be able to run against new servers that implement the spec. And also new documents can be written with the current version of the spec that run against old, un-upgraded servers (potentially with an error thrown for new features that the old server does not implement).

[rivantsov]

I want to re-iterate on important point about my experiment with Input-as-Output change, that made Graphiql fail. This got a bit misunderstood I think, and at the end we were discussing it as breaking change that breaks introspection queries for old clients and old apps. And Matt was proposing some hypothetical auto-fixes like automatically generating identical input and output types etc.

The Graphiql introspection query failed when it encountered an actual app model that uses Input type as output field type. For all old models/apps that do not use this explicitly, the updated server works OK with Graphiql. All introspection queries return identical data as before; I mentioned previously that Input and Object types still live in separate collections in introspection; nothing changes for old models and clients.

Therefore, no old clients or apps will be broken if servers start supporting these features, everything works the same. It is only app dev starts using it in his/her model, it might break Graphiql - but then, no choice; if you want to use new features in your server app you gotta upgrade server framework and clients; and let know external clients about the change you are doing - but this is a conscious choice of the app developer.

So I believe that Input-as-Output feature is still OK, doable, without breaking backward compatibility.

[benjie]

if you want to use new features in your server app you gotta upgrade server framework and clients; and let know external clients about the change you are doing - but this is a conscious choice of the app developer

@mjmahone explained really well last night why this isn’t the case. You should be able to update your schema to use newer features (so that new apps can leverage that functionality) without breaking established functionality of existing tools and apps. He also mentioned how Meta Platforms have systems that haven’t changed for years that should not break if their schema is updated to leverage newer features. For example when subscriptions was added, even if you adopted it in your schema, older clients were not impacted because they didn’t even know it existed (their introspection queries did not query the subscriptionType field) so couldn’t be confused by the introspection results.

[rivantsov]

thank you. Now it's really clear for me. Really. And that explains a lot. I think with this interpretation of backward compatibility - no chance for any meaningful progress. I am outa here. Good luck with structs. Thank you again.