Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chunk for Text #6327

Closed
harrysolovay opened this issue Jan 2, 2025 · 5 comments
Closed

chunk for Text #6327

harrysolovay opened this issue Jan 2, 2025 · 5 comments

Comments

@harrysolovay
Copy link
Contributor

I believe we should add a chunk util within text to complement the chunk util in collections. The signature would look as follows.

function chunk(text: string, size: number): string[]

Let me know if this is desirable & I'll submit a PR :)

@0f-0b
Copy link
Contributor

0f-0b commented Jan 2, 2025

What would chunk("𝐯̂", 1) return?

@harrysolovay
Copy link
Contributor Author

I would imagine ["𝐯̂"], unless, does 𝐯̂` for some reason count as multiple chars? If there's some unicode reason... this could cause issues. Maybe there'd be a third parameter enabling the user to specify how to treat cases where the cutoff is in the middle of a character.

@timreichen
Copy link
Contributor

What is the use case? Why not just split the string into an array and use collections/chunk?

@harrysolovay
Copy link
Contributor Author

That's reasonable. Especially given the aforementioned edge case.

@harrysolovay harrysolovay closed this as not planned Won't fix, can't repro, duplicate, stale Jan 2, 2025
@BlackAsLight
Copy link
Contributor

I would imagine ["𝐯̂"], unless, does 𝐯̂` for some reason count as multiple chars? If there's some unicode reason... this could cause issues.

JavaScript uses UTF-16 for its strings and this character has a length of 2. Iterating through it via its length would cause you to split the character in half with most likely gibberish on both sides until it's placed together again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants