Questions about *checkpoints* and *pretrain weights*

[yaotingwangofficial]

Dear Authours,

Many thanks for your efforts on this great work and releasing the code!

I have some questions that (I guess others may also be interested in):

Q1. how to load the pretrain weights correctly?

(1) I got the pretrain weight from: https://huggingface.co/jiayuanz3/MedSAM2_pretrain/tree/main .
(2) I modified your train_2d.py script for evaluation. Specifically, I retained only the validation part and removed the training sections.
(3) I loaded the pretrain weights with code:

ckpt_path = './checkpoints/MedSAM2_pretrain.pth'
checkpoint = torch.load(ckpt_path)
net.load_state_dict(checkpoint['model'])

net.eval()
tol, (eiou, edice) = function.validation_sam(args, nice_test_loader, epoch, net, writer)

• however, the results on REFUGE is Total score: 1.5096757411956787, IOU: 0.0159607185616769, DICE: 0.026624170053027436.

(4) I also tried with using the args.pretrain without net.load_state_dict in the above:

# with -pretrain MedSAM2_pretrain.pth
Total score: 0.6400713324546814, IOU: 0.07327658690868426, DICE: 0.10690500849569073

# without -pretrain
Total score: 0.6306238174438477, IOU: 0.09716492249995469, DICE: 0.1419979241103477

I guess my results are incorrect. May I get any guidance from you?

Q2. Different SAM2 foundation size

• Are the released pretrained weights MedSAM2_pretrain.pth suitable for different SAM2 foundation sizes? I noticed that you only included tiny and small in the code. Would it be feasible if I directly replace them with base or large variations?

• actually I tried to use load_state_dict for small size but got incompatible parameter dimensions.

[yaotingwangofficial]

check #9