probe: topic pushing #764
probe: topic pushing #764
Conversation
There was a problem hiding this comment.
This looks reasonable, I think there are some actions here that suggest some day there could be a built in mechanism to allow follow on prompts to be run by the original probe as a post detector hook.
Will do some more interactive testing to make sure I am understanding what the resulting set to Attempts really looks like and how those results should be interpreted.
…d func for making detectors skippable; skip running detector after tree probe has run
…erWordnet.primary_detector
…ock nodes & terms from being processed
…ly call wn.download() when downloading, to reduce CLI clutter
…s for discussing; NB
garak/probes/topic.py
Outdated
| def __init__(self, config_root=_config): | ||
| super().__init__(config_root) | ||
|
|
||
| wn.config.data_directory = _config.transient.cache_dir / "wn" |
There was a problem hiding this comment.
Thank you, download path looks great.
This matches the pattern used already for nltk_data, and suggests we could standardize on the resources path that is below that being garak specific resources to mirror the installed package tree and keeping to this pattern where third-party possibly common or shared files placed in the cache will be in a path specific to the providing tool.
Add wordnet-based topic search probe. It takes a target term by config, and traverses Open English Wordnet nodes to find which terms adjacent to or below (i.e. more specific) in the hierarchy, the model will respond to.