How do we use the CharacterTextSplitter module?

[wlyinan]

I want to provide GPT 4 with some long .txt inputs, and split it, pass it section through a set of prompts, then pass the combined output to a second instance of GPT 4.

It sounds pretty basic, but I can't figure out how to use the CharacterTextSplitter:

Langflow doesn't seem to allow me to connect that module's output to anything. The purple connector on the right of the module doesn't seem to fit into any flows (see screenshot).

Would anyone be able to help? Thank you!

[svupper]

Did you try connecting it to Chroma ?

[ibiscp]

Yes, the output of a CharacterTextSplitter goes to a VectorStore, try a flow like this.

[wlyinan]

Thanks so much! What if I am not intending to use it with embeddings? What I'm trying to do is: 1. Upload a longer text file 2. Have it split into 2 if it exceeds a certain number of tokens 3. Run a set of prompts on each half of the file 4. And then run another prompt on the combined output of 3. Is this the right use case for the charactertextsplitter module?

…

On Wed, 19 Apr 2023 at 21:59, Ibis Prevedello ***@***.***> wrote: Yes, the output of a CharacterTextSplitter goes to a VectorStore, try a flow like this. [image: Screenshot 2023-04-19 at 10 58 54] <https://user-images.githubusercontent.com/19699606/233098516-f4ba9b17-20fa-4294-a939-1659fdf4dfb7.png> — Reply to this email directly, view it on GitHub <#171 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADUMXTNE3SP2CTM4DQDZSELXB7VVBANCNFSM6AAAAAAXCGXA5M> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rhuanbarros]

i have the same use case

[bschleter]

hate to hack the discussion, but charactertext splitter related- how would I connect multiple data loaders to one text splitter? I am unable to do this. Thus, if I have 5 document loaders, I have to have 5 different text splitters, instead of just one.
Similarly, i am unable to have just one Chroma box that connects to multiple text splitters.

[Cristhianzl]

Hi! 👋

We are using other channels as our official means of communication with users. We apologize for the delayed response.
For faster assistance, please direct your questions to our Discord channel or open a GitHub issue, as we are much more active on these platforms.

Langflow Discord Server

Thank you for your understanding.

Best regards,
The Langflow Team