Cookbook: Processing a random or specific set of objects

A good way to both test your MIK configuration and to get a sense of the consistency of your source metadata and content is to tell MIK to process a random sample of source objects. All you need to do is use the Random Set fetcher manipulator. You tell it how many objects you want in your sample and it picks a set of that size at random.

Or, if you know of some source objects that will serve as good test data for your configuration, you can tell MIK to process only those objects using the Specific Set fetcher manipulator. This manipulator is also useful if some of the objects you generated contained errors and you want to regenerate them.

These two fetcher manipulators can be used together to generate a random set and then "replay" it, e.g., during A/B testing of MIK code/configuration, for doing demos, having a consistent set of input during speed profiling, etc. To do this, specify a file that the RandomSet manipulator should write the IDs of the randomly selected records to, then use that file as the input for the SpecificSet manipulator.

Cookbook table of contents