Generate related posts based on tags. Sort by the number of shared tags.
- Read the posts JSON file.
- Iterate over the posts and populate a map containing:
tag -> List<Post>
that has that tag - Iterate over the posts and for each post:
- Create a map:
Post -> int
to track the number of shared tags - For each tag, Iterate over the posts that have that tag
- For each post, increment the shared tag count in the map.
- Create a map:
- Sort the related posts by the number of shared tags.
- Write the top 5 related posts for each post to a new JSON file.
./run.sh go | rust | python | all
Language | Time (avg) | Time (minus I/O) | Details |
---|---|---|---|
Rust | 4.5s | - | Initial |
Rust v2 | 2.60s | - | Replace std HashMap with fxHashMap by phazer99 |
Rust v3 | 1.28s | - | Preallocate and reuse map and unstable sort by vdrmn and Darksonn |
Rust v4 | 0.13s | - | Use Post index as key instead of Pointer and Binary Heap by RB5009 |
Rust v5 | 52ms | 38ms | Rm hashing from loop and use vec[count] instead of map[index]count by RB5009 |
Rust v6 | 36ms | 23ms | Optimized Binary Heap Ops by scottlamb |
Rust Rayon | 22ms | 9ms | Parallelize by masmullin2000 |
⠀ | ⠀ | ⠀ | ⠀ |
Go | 1.5s | - | Initial |
Go v2 | 80ms | - | Add rust optimizations |
Go v3 | 70ms | 56ms | Use goccy/go-json |
Go v3 | 55ms | 34ms | Use generic binaryheap by DrBlury |
Go v4 | 50ms | 26ms | Replace binary heap with custom priority queue |
Go Con | 33ms | 10ms | Go concurrency by tirprox and DrBlury |
Go Con v2 | 29ms | 5ms | Use arena, use waitgroup, rm binheap by DrBlury |
⠀ | ⠀ | ⠀ | ⠀ |
Python | 7.81s | - | Initial |
Python v2 | 1.53s | 1.35s | Add rust optimizations by dave-andersen |
Numpy | 0.85s | 0.57s | Numpy implementation by Copper280z |
⠀ | ⠀ | ⠀ | ⠀ |
Crystal | 96ms | 50ms | Inital w/ previous optimizations |
Crystal | 72ms | 33ms | Replace binary heap with custom priority queue |