[Bugfix] Fix a bug in filtering checkpoints

[yaox12]

What does this PR do ?

When we cannot find self.monitor in checkpoint, checkpoint.find(self.monitor) should be -1 so index should be len(self.monitor) according to L98.

With the current code, checkpoints such as mp_rank_00_model_weights.ckpt would go to the True condition and cause an error in Line 103 because it cannot find the correct value to be converted to float.

NeMo/nemo/utils/callbacks/nemo_model_checkpoint.py

Lines 98 to 103 in 5c3ed94

	index = checkpoint.find(self.monitor) + len(self.monitor) + 1 # Find monitor in str + 1 for '='
	if index != -1:
	match = re.search('[A-z]', checkpoint[index:])
	if match:
	value = checkpoint[index : index + match.start() - 1] # -1 due to separator hypen
	self.best_k_models[checkpoint] = float(value)

.

Collection: [Note which collection this PR will affect]

• NeMo/nemo/utils/callbacks/nemo_model_checkpoint.py

Changelog

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Should fix [Bug] A bug in filtering checkpoints #6796

[titu1994]

Thanks !

[github-actions]

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

[yaox12]

@titu1994 Can I get this PR merged?

[github-actions]

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

[github-actions]

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.