Adding Example Annotation with Showcase Notebook

[eshwarprasadS]

Addresses #527
This PR:

• Adds an example notebook showing how to leverage SDG to enable annotation use cases.

• The example explicitly calls to guided_decoding_backend of vLLM, to make sure that the annotation options (guided_choices) are respected, while generating.

• Adds example notebook showcasing an end-to-end custom use case SDG to annotate a classification dataset using the composable components available SDG library (namely Pipeline)

[bbrowning]

This looks like a useful example to have in the repository. I rarely use Jupyter notebooks and am not really setup with a way to validate or run this to ensure it works as expected, but perhaps that's something @aakankshaduggal or @khaledsulayman can help with to give this an approval?

Also, we should think of how we keep this updated to ensure it doesn't get stale. The other examples all run as part of our unit test suite in test_examples.py. That may not be possible with a notebook in the same way, but something to think about so we know how we'll keep it working and up-to-date if we can't automatically test it.

[eshwarprasadS]

@bbrowning This is an explicit call to the guided_decoding_backend options available in vLLM (https://docs.vllm.ai/en/latest/features/structured_outputs.html).

Tested that this works, along with checking the logs from vLLM, and it seems to receive both the guided_choice and guided_decoding_backend parameters as part of the completion requests.

[bbrowning]

Great, thanks!

[eshwarprasadS]

Also, we should think of how we keep this updated to ensure it doesn't get stale. The other examples all run as part of our unit test suite in test_examples.py. That may not be possible with a notebook in the same way, but something to think about so we know how we'll keep it working and up-to-date if we can't automatically test it.

@bbrowning Thanks for the comment. You are right, that this would be a new type of artifact that would need its own testing suite. I think there are a few different ways to test notebooks in AI / ML libraries (such as ours), but something I came across that might fit our pattern could be nbmake. This plugs in to pytest and can be part of our CI and can be activated like so (?)

- name: Test Jupyter Notebooks
  run: pytest --nbmake path/to/notebooks/

I think it would make sense to make this a new issue, if we have a strong desire to keep our example jupyter notebooks tested and updated regularly.

[bbrowning]

@eshwarprasadS Yes, it's fine to make a new issue to track figuring out how or if we can keep the notebooks updated.