Simple, Ultra-fast file handling utility for text deduplication.

---

Meet refine, a powerful and user-friendly tool for the process of removing duplicate lines from files. refine is designed with efficiency in mind, making file manipulation a seamless experience for users. Below is a comprehensive guide on installing and using Seek to enhance your file processing tasks.

Features 💡

• Efficient file deduplication and management.
• Includes inbuilt sorting.
• Support for diverse input methods.
• Advanced wildcard sorting with exception handling.

Installation 🛠️

To install the refine tool, you can simply use the following command.

go install -v "github.com/yourpwnguy/refine/cmd/refine@latest"
cp ~/go/bin/refine /usr/local/bin/

Usage 📘

Usage: refine [options]

Options: [flag] [argument] [Description]

DIRECT:
  refine file.txt                       (Read and write to the same file)
  refine file1.txt file2.txt            (Read from file1 and write to the file2)

STDIN:
  cat file.txt | refine                 (Read from stdin and display to stdout)
  cat file.txt | refine newfile.txt     (Read from stdin and write to a specific file)

FEATURES: (ONLY DIRECT MODE)
  refine -w, --wildcard                 (Sort all files in the directory)
  refine -we, --wildcard-exception      (Specify files to be skipped while using wildcard)

DEBUG:
  refine -v, --version                  (Check current version)

DIRECT MODE:

Using refine to read and write the deduplicated ouptut to the same file:

Using refine to read from file1 and write the deduplicated output to file2:

Using refine for wildcard sorting (-w), which sorts all files in a directory. This feature is limited to direct mode, as during the tool's development, no use case for the pipeline mode was found.

Using refine for wildcard sorting (-w), which sorts all files in a directory except for the specified exceptions. The exceptions, meaning the files to be skipped, can be provided through the -we (wildcard exception) flag with filenames comma-separated.

---

STDIN MODE:

Using refine for sorting the lines from the standard input (stdin). The deduplicated output is displayed on the terminal without modifying the original file. This method is ideal for viewing results without altering the source file.

Using the refine for sorting the lines from the standard input (stdin), and writes the deduplicated output to a new file specified as an argument. This allows users to create a new file with cleaned data while preserving the original content. Note: If the specified file already exists and contains data, it will also be sorted.

But Why Use Our Tool❓

Well, I understand that there are already popular solutions out there like tomnomnom's anew, but here's why I think refine stands out: I've taken inspiration from those tools and built refine to be even more flexible and feature-rich. It's all about making features like wildcard sorting as straightforward and powerful as possible. With refine, I've aimed to make it easy for you to manage and sort all your files exactly how you need them.

Contributing 🤝

Contributions are welcome! If you have any suggestions, bug reports, or feature requests, feel free to open an issue or submit a pull request.