When I was using the DetAny3D repository to implement monocular 3D detection, I found that this repository uses the unidepth-v2 model. I used unidepth-v2-vitl14, but it couldn't run. After checking, I found that I should use unidepth-v2old-vitl14.
I would like to ask why the model keys differ between unidepth-v2old-vitl14 and unidepth-v2-vitl14, and what are the differences between these two models?
In the following code, two versions of unidepth_checkpoint (v2-old & v2)are used. One version (v2-old)can be replaced successfully, but the other (v2)cannot be replaced.
else:
start_epoch = 0
unidepth_checkpoint = torch.load(cfg.unidepth_path, map_location=f'cuda:{device_id}')
check_list = {k:0 for k in unidepth_checkpoint.keys()}
new_model_dict = my_sam_model.state_dict()
for k,v in new_model_dict.items():
if 'sam.image_encoder.dino' in k:
if 'pixel_encoder' + k.split('sam.image_encoder.dino')[1] in unidepth_checkpoint.keys():
new_key = 'pixel_encoder' + k.split('sam.image_encoder.dino')[1]
new_model_dict[k] = unidepth_checkpoint[new_key].detach()
check_list[new_key] = 1
elif 'sam.image_encoder.depth_head' in k:
if 'pixel_decoder' + k.split('sam.image_encoder.depth_head')[1] in unidepth_checkpoint.keys():
new_key = 'pixel_decoder' + k.split('sam.image_encoder.depth_head')[1]
new_model_dict[k] = unidepth_checkpoint[new_key].detach()
check_list[new_key] = 1
my_sam_model.load_state_dict(new_model_dict)
When I was using the DetAny3D repository to implement monocular 3D detection, I found that this repository uses the unidepth-v2 model. I used unidepth-v2-vitl14, but it couldn't run. After checking, I found that I should use unidepth-v2old-vitl14.
I would like to ask why the model keys differ between unidepth-v2old-vitl14 and unidepth-v2-vitl14, and what are the differences between these two models?
In the following code, two versions of unidepth_checkpoint (v2-old & v2)are used. One version (v2-old)can be replaced successfully, but the other (v2)cannot be replaced.
https://github.com/OpenDriveLab/DetAny3D/blob/68578738dcd6c901934b3d5672fe750982fc8a24/train.py#L200
else:
start_epoch = 0
unidepth_checkpoint = torch.load(cfg.unidepth_path, map_location=f'cuda:{device_id}')
check_list = {k:0 for k in unidepth_checkpoint.keys()}
new_model_dict = my_sam_model.state_dict()
for k,v in new_model_dict.items():
if 'sam.image_encoder.dino' in k:
if 'pixel_encoder' + k.split('sam.image_encoder.dino')[1] in unidepth_checkpoint.keys():
new_key = 'pixel_encoder' + k.split('sam.image_encoder.dino')[1]
new_model_dict[k] = unidepth_checkpoint[new_key].detach()
check_list[new_key] = 1
elif 'sam.image_encoder.depth_head' in k:
if 'pixel_decoder' + k.split('sam.image_encoder.depth_head')[1] in unidepth_checkpoint.keys():
new_key = 'pixel_decoder' + k.split('sam.image_encoder.depth_head')[1]
new_model_dict[k] = unidepth_checkpoint[new_key].detach()
check_list[new_key] = 1
my_sam_model.load_state_dict(new_model_dict)