RFC 594: <compatibility> tag in extension manifest

[eddieajau]

Summary

This RFC outlines a change to the way that Joomla checks the version compatibility of extensions. This is currently done via the version attribute in the extension tag, for example:

<extension type="component" version="3.0" method="upgrade">

The value represents the minimum version of the Joomla CMS that is required for this extension. Historically this has worked well until the introduction of long and short term distributions of Joomla. There is now a need to have more granularity for compatibility checks.

New compatibility tag

A new compatibility tag will be introduced into the extension manifest.

The compatibility tag MUST include an include tag and MAY include an exclude tag.

The compatibility tag MAY include an url attribute that refers to a page the developer has prepared concerning compatibility.

The include tag and the exclude tag if provided MUST include one or more version tags.

The body of the version tag specifies the version that is supported.

The version tag MAY specify an if attribute which defines how the version is compared. Values for the if attribute can be one of the following:

• "eq" equal to (default)
• "ge" - greater than or equal to
• "gt" - greater than

The version tag MAY include a tested_date attribute indicating the date the developer tested the specific version.

The version tag MAY include a message attribute. This is generally only used when under the exclude tag.

The body of the version tag MUST be a string compatible with PHPs version_compare method. The string MUST be in the form:

X[.Y[.Z]]

where:

• X is the major version (MUST be supplied)
• Y is the minor version (MAY be supplied)
• Z is the maintenance version (MAY be supplied)

If only the X is specified, the comparison (governed by the if attribute) is made against the major version only. Any value of the minor version will be deemed compatible.

If both X and Y are specified, the comparison major version must match, and the comparison is made against the minor version. Any value of the maintenance version will be deemed compatible.

If X, Y and Z are specified, the major and minor versions must match and the comparison is made against the maintenance version .

Consider the following examples of the version tag:

<version>2</version>

The major part of the target version must be greater than or equal to 2 (if defaults to "eq"). A values such as 2.5 would pass. Values of 1.5 or 3.0.1 would not pass.

<version>2.5</version>

The major part of the target version must match 2 and the minor part of the target version must be equal to 5. Values such as 2.5.0 or 2.5.6 would pass. Values such as 2.4, 2.6 or 3.0.0 would fail.

<version>2.5.1</version>

The target version must be 2.5.1.

<version if="ge">2.5.1</version>

Both the major and minor parts of the target version must match 2.5, and the maintenance part of the target version must be greater than or equal to 1. Values such as 2.5.1, 2.5.2 and 2.5.99 would pass. Values such as 2.5.0, 2.6 and 3.5.1 would fail.

Including and excluding versions

If a match is made on any version tag within the exclude tag, the compatibility check MUST fail.

If a match is made on any version tag within the include tag, the compatibility check WILL pass providing it has not already failed due to the exclude tag.

Consider the following examples.

<extension>
  <compatibility>
    <include>
      <version>2.5</version>
      <version>3</version>
    </include>
    <exclude>
      <version>2.5.1</version>
    </exclude>
  </compatibility>
</extension>

In this example any value of 2.5.* is compatible except 2.5.1, and any value of 3 is compatible.

General usage

The most common initial usage will be to establish whether a Joomla 2.5 web site can be upgrade to Joomla 3. An extension developer will generally put in the following compatibility tag if they are confident the extension works on both 2.5 and any version of 3. This provides the most permissive, practical option if the developer.

<extension>
  <compatibility url="http://example.com/compatibility.html">
    <include>
      <version>2.5</version>
      <version>3</version>
    </include>
  </compatibility>
</extension>

The disadvantage of the previous code is that it does elevate the risk of the user. For developer that does not want users to automatically upgrade to Joomla 3.1 without them having a change to check the code, they should consider the following:

<extension>
  <compatibility url="http://example.com/compatibility.html">
    <include>
      <version>2.5</version>
      <version>3.0</version>
    </include>
  </compatibility>
</extension>

In the event you need to specify a minimum current version, but still allow upgrading to the next minor version, the following strategy could be used:

<extension>
  <compatibility url="http://example.com/compatibility.html">
    <include>
      <version if="ge">2.5.7</version> <!-- version 2.5.x up to less than 2.6 -->
      <version if="ge">3.0.3</version> <!-- version 3.0.3 up to less than 3.1 -->
      <version if="ge">3.1</version> <!-- version 3.1 up to less than 4 -->
    </include>
  </compatibility>
</extension>

The following complex example could be typical when nearing the release of Joomla 4.

<extension>
  <compatibility url="http://example.com/compatibility.html">
    <include>
      <version if="gt" tested_date="2013-06-06">2.5.15</version> <!-- Ok for Joomla 2.5.15 and up -->
      <version if="gt">3.1.2</version> <!-- Ok for Joomla 3.1.2 to less than 3.2 -->
      <version if="gt">3.2.1</version> <!-- Ok for Joomla 3.2.1 to less than 3.3 -->
      <version if="gt">3.5</version> <!-- Ok for Joomla any version of 3.5 to less than 4 -->
      <version>4</version> <!-- Ok for Joomla any version of 4 -->
    </include>
    <exclude>
      <version>2.5.17</version> <!-- buggy version -->
      <version>2.5.18</version> <!-- major security hole that affects this extension -->
      <version>3.2.2</version> <!-- buggy version -->
      <version>3.3</version> <!-- the whole 3.3 series is excluded -->
     <version>3.5.4</version> <!-- buggy but have not found a fix for it yet  -->
    </exclude>
  </compatibility>
</extension>

For developers that cut different packages for different versions of Joomla, the following strategy could be used for a 2.5-only extension:

2.5 3

[nikosdion]

+1

[nicksavov]

This type of thing should only be required in 4.0.0 in my opinion, rather than 3.1. I don't want to see extensions not working in 6 months because of something like this. Until 4.0.0, the core should first check for the presence of the compatibility tag. If there isn't one, then it should default to the version tag. This way, we can make it optional until 4.0.0 and also move things forward. Furthermore, with the pre-upgrade check feature, we'd get more and more extensions to implement the compatibility tag without being forced to and without extensions breaking. Also, we can talk to the JED about requiring the compatibility tag for new extensions on the JED to further move things along.

Also, I would strongly advise against requiring an update every 6 months. It should instead just be once per major version and should be interpreted as the pre-upgrade check:

3

See:
http://docs.joomla.org/Pre-upgrade_Check_Version_Specification_Debate#Original

Joomla's release schedule only allows for backwards compatibility changes every major version.

Ok, with that being said, +1 on the general concept. Thanks for putting this together Andrew! :)

[eddieajau]

Updated:

• version number must be compatible with version_compare
• added note about what happens with 3.2

@nicksavov I DO want you to change your manifest every 6 months. That is deliberate and it tells me that you are a responsible vendor and have checked your code on the new major/minor version of Joomla (rather than waiting for your users to send you nasty emails). That is the purpose of this RFC. Other RFC's could have a different purpose.

It also allows for a rare condition where something changes in 3.2 that affects a subset of users. This could be a security issue, it could be a nasty bug found in 3.0.x but they put it off to fix in 3.1 to allow developers time to fix their code.

In terms of implementation though, I would still give the user the option of installing despite the fact the compatibility tags suggest otherwise (at their own risk of course). How that happens in the CMS is outside the scope of this RFC (and I've just split this out so it's easy for people to see what the debate is over). If I were you, I'd post another RFC along the lines of the wiki page you posted and then put both (maybe others will come) out for wider comment (blog post on community.j.org).

[nicksavov]

Sorry, I misunderstood github. I thought this was a pull request :) RFC = request for comment? So this is only for discussion? Similar to the mailing lists?

Could you explain how this proposed change would be in keeping with Joomla's development strategy, specifically "Every effort will be made to make sure that each release of Joomla is backward compatible with the previous minor release." of http://developer.joomla.org/strategy.html#backwardcompatibility ?

[eddieajau]

I posted this here because teasing out the examples from the CMS list discussion was difficult, and I like working with github and markdown.

The changes are in keeping with the development strategy. However, we also know many developers don't follow the "Joomla way" and it's only a matter of time until their stuff screws up. There are any number of reasons why an extension could "break" when going from, for example, 3.1 to 3.2 and it may not have to do with backward compatibility. The point of the RFC is, in addressing compatibility checks, to force the developer to rate their extension for each minor version. This IS a reasonable expectation a Joomla CMS user might have.

Also, consider this. You have an extension. It's popular but your major competitor is far more popular - no matter what you do the #1 spot on the JED eludes you. However, you know your competitor is lazy and slow to fix things, despite the popularity. Mandatory version checks allow you to get ahead in the game because all of a sudden your competitor's extension won't install. The user goes to the #2 choice which is you :) A fictitious story but you should get my point about how this could work in your favour, provided you are willing to put the work in.

[eddieajau]

Added variation A for evaluating the minor versions strictly or not.

[beat]

Hi Andrew,
Thanks for having rewritten this google-groups discussion nicely in this a bit more formal environment. Thanks also for the inclusion of the "exclude" attribute suggestion.

👍 for your proposal, with following comment:

Regarding strict or not strict, that's a decision that the extension developer can make better than the site admin, imho.

E.g. Some Joomla bugs may be deadfull show-stoppers for an extensions, while others bugs give only smaller annoyances.

So suggesting following small addition: optional "compliance" attribute with "strict" and "recommended" possible values (default value when attribute is not present, is "strict"):

     <version compliance="strict">2.5.3</version>
     <version compliance="recommended">2.5.6</version>
     <version compliance="strict">3.0</version>
     <version compliance="recommended">3.0.2</version>
     <version compliance="strict" exclude="true">3.0.1</version>
     <version compliance="recommended" exclude="true">3.0.4</version>

The example above lines means:

• v >= 2.5.3 && v < 2.6 needed mandatorily
• v >= 2.5.6 recommended
• v >= 3.0 && v < 3.1 needed mandatorily
• v >= 3.0.2 recommended
• v = 3.0.1 not compatible, won't install or allow Joomla upgrade to that version
• v = 3.0.4 not recommended, but accepted

Which would be the formal translation of this statement:
The extension is compatible with J 2.5.3+ (2.5.6+ recommended) and 3.0+ (3.0.2+ recommended - except 3.0.1 which is incompatible and 3.0.4 which is not recommended).

The nice thing is that above formal XML description could be automatically translated into the English statement just above.

What do you guys think of this suggested addition comment ?

Best Regards,
Beat

P.s. (not a blame, just a suggestion:) Btw, for scientific and community fairness, a reference with link to the Google-group thread would be nice to add to your RFC, as this RFC is the result of a long discussion with many small and large contributions from others. I am not adding the URL in this comment, as it should imho be comming from you at top of your RFC, e.g. as introduction ? :)

[eddieajau]

The discussion thread is really hard to follow; I don't think there's much point in an outbound reference for the moment. I'm not hearing enough support for people wanting to work out the specification first so I think we'll just park this for now. Any forward motion is going to be difficult until we can actually qualify what the users want.

Regarding the compliance attribute, I think it's an interesting idea - I'd have to think about it some more.

[eddieajau]

Updated the RFC following ideas from Brad Gies.

[beat]

👍

[mahagr]

Is there a minor typo when trying to explain (3.0.1 passes):

<version>2</version>

I think this would be a great addition and concerning @nicksavov fears, I suggest that we add the same feature to the updater xml files sitting in the vendor server. Basically if the compatibility information was provided in the updates, it would override those provided in the manifest file. It also has benefit of preventing upgrade to a minor version -- let's say that Joomla! 2.5.9 got nasty bug that breaks the component -- without needing to send email to each user suggesting not to upgrade. The feature also helps in component upgrades as the compatibility check is made before component gets downloaded to the server.

While we are on it -- would it be cool to have the same compatibility checks between extensions. For example @beat and I could benefit from extending the feature a bit.

@eddieajau Why the issue has been closed?

[beat]

@mahagr : great ideas!

1. I love the idea that the update-checker also updates automatically the compatibility information in xml format ! That could go into a file, or better, as changing configuration files of extensions is not recommended (breaks sanity-files-CRC-checks), in a new column of the update-checker database table 👍

2. yes, for the idea of cross-extension compatibility checking 👍
   in that case, it would make sense to have multiple tags and an attribute for each for what it is.

For now, all compatibility towards joomla could look like (and joomla could look for 'compatibility[@towards="joomla"]' (xpath expression):

<compatibility towards="joomla">

And for instance, CB could have:

<compatibility towards="joomla">
...
</compatibility>
<compatibility towards="com_kunena">
....
</compatibility>
....

That way, if Kunena is installed, it would check the version of Kunena too. And the other way around, Kunena could have a compatibility information towards CB.

(syntax above is just for example, variations welcome).

[bgies]

I second this thought @mahagr : great ideas!

It's why I've been passionate towards making this spec as good as possible. I can see all kinds of very useful things coming from it. I even mentioned early on in the Google Groups thread that I was 100% sure hundreds of developers would find uses for this. Happy to see concrete examples from well-established developers though. It makes it a lot easier to make that case :).

[bgies]

I just sent a message for Andrew on the Google Groups thread asking if anyone thinks it's a good idea to add an optional message attribute on the version tag, and that would tie in with Beat's compliance tag or remove the need for it.

2.5.6

OR 

2.5.6

[bgies]

Ooops it ate my code

<include>
<version message="Works on 2.5.6 except that 2.5.6 has a bug that prevents the "something page" from displaying correctly. Recommend updating to 2.5.7">2.5.6</version>
</include>
OR 
<exclude>
<version message="Works on ALL 2.5.x versions except 2.5.6 has a bug that breaks everything. Recommend updating to 2.5.7">2.5.6</version>
</exclude>