RFC 598: <compatibility> tag in extension manifest (versions-from-to)

[eddieajau]

Summary

This RFC outlines a change to the way that Joomla checks the version compatibility of extensions. This is currently done via the version attribute in the extension tag, for example:

<extension type="component" version="3.0" method="upgrade">

The value represents the minimum version of the Joomla CMS that is required for this extension. Historically this has worked well until the introduction of long and short term distributions of Joomla. There is now a need to have more granularity for compatibility checks.

Specification

A new compatibility tag will be introduced into the extension manifest.

The compatibility tag MUST include an include tag and MAY include an exclude tag.

The compatibility tag MAY include an url attribute that refers to a page the developer has prepared concerning compatibility.

The include tag and the exclude tag if provided MUST include one or more versions tags.

The body of the versions tag may be supplied acting as a description for the tag.

The versions tag MUST include either a from attribute, or a to attribute, or both. The value of from or to MUST be in a form recognised by the PHP version_compare method. If both are supplied, the value of to MUST be equal to or greater than the value of from.

If supplied, the from attribute nominates the lowest value for version comparison (that is, literally "from" version X.Y.Z).

If supplied, the to attribute nominates the highest value of the version comparison (that is, literally up "to" and including version X.Y.Z).

If a match is made on any versions tag within the exclude tag, the compatibility check MUST fail.

If a match is made on any versions tag within the include tag, the compatibility check WILL pass providing it has not already failed due to the exclude tag.

Examples.

Consider the following examples.

<extension>
  <compatibility>
    <include>
      <versions from="2.5" />
    </include>
  </compatibility>
</extension>

This reads "versions from 2.5 up" will be compatible (including version 3, 4 and any other future versions).

<extension>
  <compatibility>
    <include>
      <versions to="3.0.99" />
    </include>
  </compatibility>
</extension>

This reads "versions up to 3.0.99" are compatible (effectively anything less than version 3.1).

<extension>
  <compatibility>
    <include>
      <versions from="2.5.2" to="2.5.99" />
      <versions from="3.0.1" to="3.0.99" />
    </include>
    <exclude>
      <versions from="3.0.3" to="3.0.3" />
    </exclude>
  </compatibility>
</extension>

This reads "versions from 2.5.2 to 2.5.99" are compatible, and so are "versions from 3.0.1 to 3.0.99", but "version 3.0.3" is not compatible.

The following is recommended for dual 2.5+3.x extensions:

<extension>
  <compatibility>
    <include>
      <versions from="2.5.8" to="2.5.99" />
      <versions from="3.0.0" to="3.99.99" />
    </include>
  </compatibility>
</extension>

or

<extension>
  <compatibility>
    <include>
      <versions from="2.5.8" to="2.5.99" />
      <versions from="3.0.0" />
    </include>
    <exclude>
      <versions from="4.0.0">Not ready yet</versions>
    </exclude>
  </compatibility>
</extension>

[bgies]

I'd be happy with this. I think it includes all possible scenarios, and it is easy to read. Can we make it mandatory that the compatibility tag contains the extension attribute? If the default is joomla and some developer releases an extension and forgets to put the extension tag in, and it's meant for another extension, it could cause all sorts of havoc :).

So... mandatory to be this format for the compatibility tag:

<compatibility extension="joomla">

[eddieajau]

It doesn't worry me, I'm just a bit vague about how the extension attribute is intended to be used. Like, is that the CMS's responsibility to do the cross-checking? I just feel we are trying to solve a different problem that hasn't really been defined yet.

Does it mean there can be multiple compatibility tags in an extension manifest? If so, there should compatibility reside in a container itself?

[betweenbrain]

Regarding requiring the extension attribute, wouldn't a non-Joomla extension need to contain a properly formatted XML file that meets these, and other requirements, to run the risk of being installed into the Joomla CMS to cause havoc? It seems like that happening would be a rare occurrence, if ever. If this was for the Platform, I would think differently, but my opinion is that requiring it adds a, albeit slight, unnecessary complication.

[bgies]

Well, the way Beat used it, it would mean multiple compatibility tags, but that can be decided later. I don't think we have to solve that problem now, just accommodate it. Maybe just change this tag name from compatibility to joomla_compatibility, and leave the compatibility tag for the other use, as it will require an attribute to specify the extension? Or they could extension_compatibilty for their tag name. Obviously, they'd want to use the same functions and rules if possible, but the rules above should be no problem.

[betweenbrain]

I'd personally prefer to see <compatibility> mean Joomla CMS, and other extensions could use <compatibility extension="foo"> . I think that would accommodate other extensions while simplifying this for the majority use case.

[eddieajau]

I'd stick it under a dependancies tag if they want to do that.

<dependancies>
   <extension name="com_foo">
     <compatibility>

In hindsight, I think it's better to leave the extension attribute out completely - it's just a "tag" and doesn't have any influence on the comparison logic. Someone else can deal with it later. That said, I don't care if it stays there, I just don't want to get distracted over working out how it's used.

[eddieajau]

Ripped it out. Someone else can champion the debate of extension-to-extension compatibility.

[nicksavov]

+1 Generally looks good so far.

One of the rules is:
"The versions tag MUST include either a from attribute, or a to attribute, or both. The value of from or to MUST be in a form recognised by the PHP version_compare method. "

What's the following interpreted as?

Joomla 1.0.0 to 3.5.99?

[betweenbrain]

Works for me. Whatever the final solution, the simpler for the majority of uses, the better in my opinion.

[eddieajau]

Joomla 1.0.0 to 3.5.99?

Target version is >= 1.0.0 and <= 3.5.99

[eddieajau]

@betweenbrain right, and this is just the specification of the compatibility tag itself. The spec for it should be portable (it doesn't matter to me how or where it's buried in the extension manifest).

[betweenbrain]

@eddieajau +1

[nicksavov]

If a version is neither included nor excluded, is it interpreted as "unknown" or does it default to excluded?

[eddieajau]

If a version is neither included nor exclude, it's interpreted as "unknown", correct?

I would imagine the "check" method would return:

• true if the version is explicitly included
• false if the version is explicitly excluded
• null if the version falls outside of the previous two cases

What the UI does with a null value is for someone else to decide.

[nicksavov]

OK, the UI for the pre-upgrade check will probably be "No" (i.e. not compatible). Anyone have any objections to that?

[eddieajau]

The UI is outside the scope of this RFC :)

[eddieajau]

Actually, true or false will do. Here's what I think the interface will look like:

Ooops
https://gist.github.com/4274047

[nicksavov]

Sorry, it ripped out my code at #598 (comment)

One of the rules is:
"The versions tag MUST include either a from attribute, or a to attribute, or both. The value of from or to MUST be in a form recognised by the PHP version_compare method. "

What's the following interpreted as?

      versions to="3.5.99"

[eddieajau]

<versions to="3.5.99">

means any version less than or equal to 3.5.99.

[nicksavov]

k

[nicksavov]

What's from="4.0.0">Not ready yet</versions for in your example? i.e., what's the "Not ready yet" for? Is it an inline comment?

[bgies]

I agree that the other use should just use a different tag name; Then we don't need the attribute at all.

I'm not sure about allowing one tag to be used for multiple major versions, but I can live with it. In the end some developers will lie or be overly optimistic about their extension, but we don't gain much from forcing them to use multiple tags for it.

This spec will require a little more coding than the other one, but will be much easier for devs to use, so let's go with it.

[eddieajau]

what's the "Not ready yet" for? Is it an inline comment?

Instead of the "message" attribute we talked about in the last proposal. I just made it the body of the tag since we aren't using that anymore. The idea would be com_joomlaupdate could get that message and maybe display it in the UI.

[bgies]

The logic should be :

If (version is excluded)
return false
elseif (version is included)
return true
else return null/unknown

[beat]

@eddieajau ,
No issues if you are not interested in extension-to-extension compatibility. Just please keep somewhere a "joomla" selector (attribute content or tag), so that Joomla upgrade itself can ignore XML parts which don't have that selector, so that you allow "Someone else to champion the debate of extension-to-extension compatibility" independently.
Thanks

I also like the message for cleartext compatibility information that @bgies commented on your RFC #594 . Even if it probably should be an attribute instead of data content (many good reasons for that, outside of the scope of this discussion).

[danyj]

+1

[rasiodesign]

"This reads "versions from 2.5 up" will be compatible (including version 3, 4 and any other future versions)."

Not a very good idea, if you ask me, cause I can develop a component for version 2.5.10 and up to 2.5.99 and I'
ll add compatibility tag as







If one of my client is upgrading Joomla from 2.5.x to 3.0.x then my component will appear as compatible even if it's not.

[rasiodesign]

"This reads "versions from 2.5 up" will be compatible (including version 3, 4 and any other future versions)."

Not a very good idea, if you ask me, cause I can develop a component for version 2.5.10 and up to 2.5.99 and I'
ll add compatibility tag as

versions from="2.5"

If one of my client is upgrading Joomla from 2.5.x to 3.0.x then my component will appear as compatible even if it's not.

[subtext]

I think this looks great as is. I heartily endorse it. I do not see the need to modify this any further. There are many ways in which a developer can express their intent within this syntax.

@rasiodesign If you don't want your clients to upgrade to version 3.0, then this system gives you several ways to express it. Here are just two examples:

<extension>
  <compatibility>
    <include>
      <versions from="2.5.10" to="2.5.999" />
    </include>
  </compatibility>
</extension>

OR

<extension>
  <compatibility>
    <include>
      <versions from="2.5.10" />
    </include>
    <exclude>
      <versions from="3.0" />
    </exclude>
  </compatibility>
</extension>

[danyj]

SIlviu might have a point but solution could be force the to= attribute , this way devs should define what exact from-to versions they support and this would not result in versions bugs . Now if they are shady to do from="2.5.0" to="7.0.999" that is completely something else and should not fall back on us.

[subtext]

@rasiodesign

If a developer makes a bad decision that is bad for his or her clients, then the client will pay the price. Also, maybe the developer will learn a lesson. It should not be for us to stop developers from making bad decisions. I don't want to look over your shoulder and make sure you do it the "right" way. That is up to you. If some other developer makes a bad decision, that hurts their clients, not mine. I can't stop anyone from making bad decisions.

[rasiodesign]

@subtext

you can stop him by making the
versions from="2.5.10"
compatible for 2.5.10 and UP before 3.0.0 and not the rest of the Joomla versions.

I remember that we agree with Nick that this will be the functionality and not is changed :) and the worst think it that is bad now

[bgies]

@beat - I think it would be good if you could give us a (short) summary of why an attribute would be better than the value for the message :). We're using attributes anyway, so if there are good reasons for attributes, it would be much better to change that now.

Also, one thing that has been nagging at me is that this spec doesn't have the optional test_date attribute, which we had in 594. Not sure if Andrew deliberately dropped it (the dev could easily put in the message), or it was just overlooked. I'd like to see it included because when the UI (there will probably be many different versions in many extensions) is designed it would give the UI designer the opportunity to compare the test date with the current date and give more information to the user. I'm happy without it, but I think it might improve the user experience, and encourage extension devs to update more frequently, making it easier for users to choose actively developed extensions instead of one-time abandoned extensions.

[bgies]

My preference would be to make both the "from" and "to" attributes mandatory for each version tag. That still allows devs to include multiple major versions in one single tag, but forces them to declare their intentions, and is less prone to devs making errors.

It works as is, but would be a little better with mandatory "from" and "to" attributes.

[rasiodesign]

@bgies

exactly

[eddieajau]

You'll probably want to leave that feedback in the forum.

[jmcameron]

I like the option of including a comment in the exclude part:

<exclude>
  <versions from="4.0.0">Not ready yet</versions>
</exclude>

Will the standard also allow similar comments on the 'include' part? Eg, to allow the extension developer to indicate what platforms it was tested on, etc.

[bgies]

I'm seeing support for checking compatibility with other extensions (and I agree it would be very useful). Shumisha on the forum suggested this

   
<compatibility>
  <joomla>
       <exclude>
            <versions from="4.0.0">Not ready yet</versions>
       </exclude>
  <joomla>
</compatiblity>

I think that works well for Joomla because the tag is known, but, I don't think it would work as well for other extensions, so my suggestion is that it be implemented as a attribute. @beat suggested like item="joomla", and I like that so :

   
<compatibility>
  <with="joomla">
       <exclude>
            <versions from="4.0.0">Not ready yet</versions>
       </exclude>
  </with>
</compatiblity>

and other extensions would then use <with="com_comprofiler"> or <with="com_kunena">

[subtext]

@bgies I like this idea very much. I think this would go a long way to solving the compatibility issues that "most" users are experiencing. So many of these issues are only 3rd party related. This would be a great way to cross check against other installed extensions that are frequently used on joomla sites. It seems like a great regulating mechanism.

[eddieajau]

@bgies I'd still keep scope modifiers out of the compatibility tag (that is <scope><compatibility>..., not <compatibility><scope>...). It keeps the tag atomic and portable.

[shumisha]

Hi
To clarify my proposal, this is NOT intended for extension dependencies, but simply to allow the Joomla installer, in the future, to apply the same sort of checks before installing an extension that's envisioned now for the Joomla updater.
Things that could be checked are php version, database type and version for instance

@andrew: within an extension manifest I think the "scope" is actually "compatibilty", so I think it makes sense to have a wrapping tag for this group of tags. ie:

<extension>
    <files folder="site">
    <filename>controller.php</filename>
    <filename>index.html</filename>
    <filename>metadata.xml</filename>
    </files>
    <languages folder="site">
    <language tag="en-GB">language/en-GB.com_xxxxxx.ini</language>
    </languages>
    <compatibility>
                <with name="joomla">
                        <include>
                                  <versions from="2.5" />
                        </include>
                </with>
                <with name="php">
                        <include>
                                  <versions from="5.3.8" />
                        </include>
                </with>
  </compatibility>
</extension>

[eddieajau]

@shumisha In my opinion, the compatibility tag itself should be kept as simple and portable as possible. It should answer the question "is the version compatible with the target version?". It should not care whether that's the CMS or any particular extension. Defining the application scope should be "outside" that tag so that the compatibility tag can be passed to a generic version compare function.

[shumisha]

Andrew the point of the proposal is that this format specification should not allow only one compatibility tag but a series of them, hence the need for a "grouping" tag, so as to avoid having multiple compatibility tag spread throughout the "global" extension tag
I have reformated my example below, as the xml markup would not appear at my first attempt

An alternative was also proposed similar to:

    <compatibilities>
        <compatibility with="joomla">
               <include>
                      <versions from="2.5" />
                </include>
        </compatibility>
        <compatibility with="php">
               <include>
                        <versions from="2.5" />
                </include>
        </compatibility>
    </compatibilities>

which is fine by me, as long as the format allows having multiple compat checks specification.

I introduced a very similar version update check (with min, max, include and exclude version numbers) for my extensions more than 2 years ago, and it took only a few months for me to realize I had to check several things, and so I needed multiple tags.

Rgds

[eddieajau]

@shumisha Yes, that's fine. That's keeping the "scope" out of the code that actually does that compatibility check.

[beat]

👍 on the

<compatibilities>
    <compatibility with="joomla">

markup proposal, thanks @shumisha for proposal and @eddieajau for acceptance of this comment to this RFC.

[jmcameron]

There is an underlying assumption that developers will keep there applications up to date and site managers will update their extensions regularly. Part of the goal, as I understand it, is to encourage developers to keep things up to date, which is reasonable.

What about the case where a site manager might want to determine if your extension is compatible with the site BEFORE they install it. I wonder if it makes sense to have this compatibility information available outside the extension installation files...like perhaps on the extension update server? With the given information, it is not hard imagining that a compatiblity check could tell you YES (it is compatible) or NO (it is not compatible along with the reasons why).

[eddieajau]

@jmcameron that's a really great comment. My thoughts on that are you'd add a feature to the installer where you can do a dry-run compatibility check before installing.

[mahagr]

@jmcameron, I already made similar comment in #594:

I suggest that we add the same feature to the updater xml files sitting in the vendor server.

Basically if the compatibility information was provided in the update server, it would override those provided in the manifest file. It also has benefit of preventing upgrade to a minor version -- let's say that Joomla! 2.5.9 got nasty bug that breaks the component -- without needing to send an email to every user telling them not to upgrade or hoping that they check the forum before doing that. The feature also helps to make component upgrades faster and more robust as the compatibility check is made before component gets downloaded.

[roland-d]

After having read all posts, I would like to express my support for this feature and I see a lot of use for it. I see it being used as below (extended on Shumisha example):

    <compatibilities>
        <compatibility with="joomla">
               <include>
                      <versions from="2.5" />
                </include>
        </compatibility>
        <compatibility with="php">
               <include>
                        <versions from="2.5" />
                </include>
        </compatibility>
        <compatibility with="mysql">
               <include>
                        <versions from="5.1" />
                </include>
        </compatibility>
        <compatibility with="com_foobar">
               <include>
                        <versions from="1.3" />
                </include>
        </compatibility>
    </compatibilities>

The comment made by jmcameron is also a very good one, the dry-run option will save a lot of headache.

[mariopro]

I've been reading the posts that have arrived so far, and would like to share mho on this subject. The compatibility tag looks very useful as long as it doesn't go beyond the simple:
<compatibility><include><versions from="x.x.x" /></include></compatibility>

If is not compatible with Joomla! it shouldn't be used (dot) ... otherwise the manifest will have more compatibility tags than the actual important ones - those that are the core of Joomla! principle of logic, design and concept. Components, modules, templates, whatever is available out there for Joomla!, should be compatible with Joomla!, because Joomla! 3 is compatible with PHP x.xx, MySQL x.xx and so on, Joomla! 2.5 the same, 1.5 the same, and so on.
With all the amount of compatibility tags proposed, or made available, we might get to a point where there will be errors or collisions between a Joomla! manifested compat version and other compat tags, or it will be a big mess. So, imho, I would say that if it's not compatible with Joomla!'s compatibilities, it shouldn't be compatible with Joomla!™.

[shumisha]

Hi mario

I find this a bit too simple on the medium term, even on short term. As as real-life example, when the first version of sh404SEF for Joomla 1.5 (J! 1.5 was compatible with PHP 4) was released, in early 2008, it already required PHP5 (not by personal choice, but to solve an issue with Joomla! pagination). Likewise, JCalpro required PHP5 very early on, to be able to properly handle timezones.

Having "collisions" will happen : compatible with joomla 3, but maybe not with postgres for instance, or require a more recent php version. That's fine, as long as a proper rule is in place.
These collisions already exist with the current, single, compatiblity tag: the include list can collide with the exclude list. In such case, the specs says that the exclusion tag wins, ie the check will fail.

However the Joomla installer choose, in the future, to handle the php, mysql, or whatever tags we come up can only be guessed at this stage. My guess is that all tags will be required to pass for the check to pass.

Another point worth mentioning again is that I don't think Joomla installer (remember that these added tags are not for the joomla updater, but for future use in the Joomla installer) should only take care, and check, its own list of core tags, and not bother with installed extensions.
Though made possible the way this specs are done, I think it'll be one bridge too far for the core. I'm also extremely doubtful developers will put things such as "not compatible with K2" or "doesn't work with sh404SEF" in their manifests ;)

Rgds

[mariopro]

Hi shumisha,
While in joomla compatibility, as stated in the opening post of this thread and the examples given, I couldn't agree more with this feature, after all, some of us are already doing this compatibility version check in installer scripts using the version_compare to properly find the correct upgrade method to be used. placing it in the manifest will add efficiency to the process, imho.

What, for me, it's difficult to understand is the non joomla compatibility checks for extensions, not for the core installer (this seems to be logical to perform). Some of the examples given here, with compatibility tags for PHP, MyQL, com_xxxxx, etc. seem to make more sense to use an than the tag ... A good example are templates that are now in the market that state J3 compatibility, responsive, blah, blah, blah, and I've seen a lot of people purchasing these templates and contacting extension developers because extensions don't work on these templates because of the frameworks that they use, that aren't even part of Joomla! core libraries.

What I'm imagining is that extension developers will just be ignoring the bunch of possible compatibility checks with third party or non Joomla! core libraries, because they are developing for Joomla! and not for whatever frameworks, libraries or scripts that templates or third party extensions may introduce to override the Joomla! core - take Gantry as an example.

This to say that I'll be a supporter with the Joomla! compatibility check, but not with whatever may be out of the Joomla! core minimum requirements. If a Joomla! version states minimum requirements PHP 5.3 developers have to keep this in mind and not state in their extensions or templates a compatibility with that J3 and PHP3, or J3 but no javascript or no jQuery. Doesn't make sense. ;)

Rgds

[shumisha]

Hi
"While in joomla compatibility, as stated in the opening post of this thread and the examples given, I couldn't agree more with this feature"

This specification is NOT being discussed for use by 3rd-party devs. This is a specification by and for the Joomla! project, to be used by the Joomla project inside its updater (current discussion) and possibly in the future in the Joomla! installer.

3rd party devs (this includes templates BTW) will then be able to tell the Joomla! CMS with which version they are compatible by adding authorized tags in their manifest. If the added "with"... sub tag is included in the spec, then 3rd party devs may also specify additional requirements they may have such as a min php version or database type or version.

That's it.

Personally, I have only been speaking of checking php and database type and version. I also thought about "bootstrap" version and "jquery" version, in case those are updated in 3.1+.
Adding tags for com_xxx does not make any sense to me.

3rd party extensions may add all the tags in the world to their manifest, the Joomla! project will read and use only those we decide for. Currently, only the "joomla version number" is included in the spec, for use by the Joomla updater. By definition, that's the only useful tag for the Joomla updater.

"If a Joomla! version states minimum requirements PHP 5.3 developers have to keep this in mind and not state in their extensions or templates a compatibility with that J3 and PHP3,"
As per the real life example I already gave above, it's more likely to work the other way around: Joomla may be ok with PHP 5.3, but for instance an extension requires PHP 5.4 because of some features it needs. That's perfectly fine, and should in no way be forbidden, as long as it's clearly stated.

Rgds

[piotrmocko]

There is one more compatibility which could be consider - Joomla Platform. I have some extensions which use methods from specific Platform version and it would not work in older version of Platforms. Of course it could defined by Joomla version but it is another point of view. I leave it for discussion if someone would like to have such feature.

[nicksavov]

Thanks everyone! This has been implemented now. Please see the announcement at:
https://groups.google.com/d/msg/joomla-dev-cms/eIGjGvSUZ3A/JQz1XtgL_IcJ

[bgies]

Thanks for letting us know Nick. It's great news.

Brad.

On 2/28/2013 4:39 AM, Nick Savov wrote:

Thanks everyone! This has been implemented now. Please see the
announcement at:
https://groups.google.com/d/msg/joomla-dev-cms/eIGjGvSUZ3A/JQz1XtgL_IcJ

—
Reply to this email directly or view it on GitHub
#598 (comment).

[nicksavov]

My pleasure :)

[nicksavov]

Marking as resolved. The code's at https://github.com/nicksavov if anyone is interested.