(+)go-url-normalize[RFC 3986]

Normalize URL | Count unique normalized URL | Count unique normalized URL per TLD

Installation

Install the package by doing go get:

go get github.com/slimdestro/gourlnormalizer

Three exported methods that you can use in your modules are:

NormalizeURL(url string) 
CountUniqueNormalizedUrls(url []string)  
CountUniqueNormalizedUrlsPerTopLevelDomain(url []string)   

Example

package main 

import (
	"fmt"
	"github.com/slimdestro/gourlnormalizer"
)
func main() { 	 
	normalizedUrl,_ := normalizer.NormalizeURL("https://example.com?b=2&a=1")
	fmt.Println(normalizedUrl)
        // compare both url to see what has changed
 
	urls__ := []string{"https://example.com?a=1&b=2", "https://example.com?b=2&a=1"}
	fmt.Println(normalizer.CountUniqueNormalizedUrls(urls__))
	 
	urls___ := []string{"https://tempo.com", "https://example.com", "https://subdomain.example.com"}
	fmt.Println(normalizer.CountUniqueNormalizedUrlsPerTopLevelDomain(urls___))
}
 

Author

slimdestro(Mukul Mishra)