docs: ✏️ add sections

[severo]

No description provided.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[severo]

Hi @stevhliu: I created a basis for the docs with the structure you recommended (along with @lhoestq) and started moving content and filling the new parts.

Do you need me to work on other things, or is it OK for you to start improving the content? Please let me know. Thanks!

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[lhoestq]

Awesome ! Thanks :)

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[stevhliu]

Hey @severo, let me know what you think of the changes so far; the only sections left to work on are the concept guides :)

Some things for consideration:

• What do you think about removing the setup page since we cover it in quickstart?
• For the preview a dataset guide, I think it'd be cool if we can embed the dataset viewer in the docs so users can see what it looks like and play around with it. Otherwise, we can just do a screenshot.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[severo]

Thanks a lot @stevhliu! It improves the docs a lot, it reads a lot easier now.

What do you think about removing the setup page since we cover it in quickstart?

Sure

For the preview a dataset guide, I think it'd be cool if we can embed the dataset viewer in the docs so users can see what it looks like and play around with it. Otherwise, we can just do a screenshot.

It could be a screenshot for now. We cannot embed the Hub viewer, it's not standalone HTML. I'll see if it's possible to generate a very simple viewer on Observable and embed it in the docs.

Looking forward for the concept guides!

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[stevhliu]

Done with a first pass of the docs, I'll work on editing it for clarity next!

1. I removed the add_tasks.mdx page because it doesn't discuss concepts behind Datasets Server and seems more like a roadmap/how to contribute. We can add a page to the guides for how to contribute, kind of similar to the Transformers docs if you'd like.

2. I think it'd also be neat to have a separate concept guide for discussing the technology/infra behind Datasets Server and how it supports other projects on the Hub. That'll give a more concrete example of how users can integrate Datasets Server into their own projects and what they can build with it.

3. Lastly, for the configs_and_splits.mdx page, I'll work on some kind of graphic illustrating the relationship between splits, configurations, and a dataset :)

[severo]

Oh, it's excellent, thanks a lot.

For points 1 and 2, you're right. I think #601 is precisely saying the same thing: it's not clear in the current public doc what the API is and what the server is. Let's try to make it more explicit upfront both in the docs and in the repo's README. Can you help with this too?

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[stevhliu]

Updated index.md to clarify the difference between the API and server in Datasets Server, and added a concept guide on how the server works (feel free to add more details here if I'm missing anything)!

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[HuggingFaceDocBuilder]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[severo]

Excellent, it's a lot clearer! And I love the GIF :)

I'll merge so that the users benefit from the new docs. If needed we will still be able to iterate on it, but it's already a huge improvement. Thanks, @stevhliu!