Skip to content
This repository has been archived by the owner on Jan 8, 2020. It is now read-only.

Automatic TLDs update for Zend\Validator\Hostname #7307

Conversation

Tauop
Copy link
Contributor

@Tauop Tauop commented Mar 11, 2015

Hi,

I improve the update script provided in #7303 :

  • in bin/ directory
  • update $validTlds definition in file library/Zend/Validator/Hostname.php
  • several errors detection

usage : php ./bin/update_hostname_validator.php

Is the script name ok ?

Patrick

@Tauop Tauop mentioned this pull request Mar 11, 2015
@moderndeveloperllc
Copy link
Contributor

My only reservation is that this script only inserts new TLDs, there is no mechanism if a gTLD is removed from the list. Might it be better to generate the entire list and simply replace $validTLDs? Of course if you want to spend the time, you could also look into the Zend\Code\Generator classes and see of that might help.

@moderndeveloperllc
Copy link
Contributor

A ha! I found what I was looking for. The Root Zone Database is also available as a daily-generated text file, though only with A names - https://data.iana.org/TLD/tlds-alpha-by-domain.txt

Which would be best to pull from? The Validator class does have its home-grown decodePunycode() function that could be used if idn_to_utf8() was not available. It would help remove the dependance on checking for specific CSS to parse TLDs from an HTML file.

@Tauop
Copy link
Contributor Author

Tauop commented Mar 11, 2015

Hello,

My only reservation is that this script only inserts new TLDs, there is no mechanism if a gTLD is
removed from the list. Might it be better to generate the entire list and simply replace $validTLDs? Of
course if you want to spend the time, you could also look into the Zend\Code\Generator classes and
see of that might help.

I don't make a diff.
I put the new list in place.

Zend\Code has so less document. Can't figure out how to use it.
Usage Zend\Code\Generator : don't know how to parse an existing class, and if the generated class will respect zf2 coding style... that's why I made a "replace" algorithme.

Patrick

@Tauop
Copy link
Contributor Author

Tauop commented Mar 11, 2015

A ha! I found what I was looking for. The Root Zone Database is also available as a daily-generated
text file, though only with A names - https://data.iana.org/TLD/tlds-alpha-by-domain.txt

Very nice !

Which would be best to pull from? The Validator class does have its home-grown decodePunycode()
function that could be used if idn_to_utf8() was not available. It would help remove the dependance
on checking for specific CSS to parse TLDs from an HTML file.

I will push a new PR using this method.

Patrick

Tauop added 2 commits March 11, 2015 15:18

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
@Tauop
Copy link
Contributor Author

Tauop commented Mar 11, 2015

@moderndeveloperllc
I update the PR, with usage of https://data.iana.org/TLD/tlds-alpha-by-domain.txt.
The good news is that we don't take into account :

  • 'test' like .テスト
  • 'country-code not assigned' TLD like .bl.

BTW, I don't understand why .oracle is present in 'text-version' of the database, but not in the 'html-version' :-/

Besides, I use Zend\Validator\Hostname::decodePunycode() is idn_to_utf8() is not present. However, I have to use reflexion to make this method accessible :-/

Patrick

@moderndeveloperllc
Copy link
Contributor

Sorry about the code generation misunderstanding. It was too late last night when I was trying to mentally parse your code! I remember when I was thinking about rewriting the validator last year I too noticed that the text list had more than the HTML list. The .oracle domain definitely exists, though I'm not sure anyone can actually buy one. It looks to be a vanity gTLD that's owned by Oracle.

I imagine that @weierophinney will want you to write a unit test for this.

@Tauop
Copy link
Contributor Author

Tauop commented Mar 11, 2015

@moderndeveloperllc
I don't see what to unittest, as this is a script you run before a release, and you see modifications made by commit Hostname validator. BTW, where do I put unit test ?

Otherwise, I have break a testcase :

1) ZendTest\Validator\HostnameTest::testAdditionalUTF8TLDs
The input appears to be a DNS hostname but cannot match TLD against known list
The input does not appear to be a valid URI hostname
The input does not appear to be a valid local network name - test123.δοκιμή
Failed asserting that false matches expected true.
/home/travis/build/zendframework/zf2/tests/ZendTest/Validator/HostnameTest.php:504

For me, test123.δοκιμή is not a valid Hostname, as δοκιμή is not listed on https://data.iana.org/TLD/tlds-alpha-by-domain.txt.
However, δοκιμή is listed on http://www.iana.org/domains/root/db as type = "test" and Sponsoring organisation = "Internet Assigned Numbers Authority".

What do you think ?
Is a 'test' TLD valid ??

Patrick

@moderndeveloperllc
Copy link
Contributor

I would say that is not a valid TLD. The root domain database page for that TLD indicates, "This domain is not present in the root zone at this time."

@Tauop
Copy link
Contributor Author

Tauop commented Mar 12, 2015

FYI, The last Tracis failed build is not due to my PR

Legend: ?-unknown, I-invalid file syntax, file ignored, .-no changes, F-fixed, E-error
   1) ServiceManager.php (empty_return)
      ---------- begin diff ----------
      --- Original
      +++ New
      @@ @@

      -        return null;
      +        return;
           }

@weierophinney weierophinney added this to the 2.4.0 milestone Mar 17, 2015
@weierophinney weierophinney merged commit b5b7a2b into zendframework:develop Mar 17, 2015
weierophinney added a commit that referenced this pull request Mar 17, 2015
…tomatic-update

Automatic TLDs update for Zend\Validator\Hostname
weierophinney added a commit that referenced this pull request Mar 17, 2015
weierophinney added a commit to zendframework/zend-validator that referenced this pull request May 15, 2015
…e/hostname-validator-tld-automatic-update

Automatic TLDs update for Zend\Validator\Hostname
weierophinney added a commit to zendframework/zend-validator that referenced this pull request May 15, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants