[FEATURE] timm.create_model() load pretrianed model from local path

[ShallJaiden]

I tried to load a model checkpoint using timm,

model = timm.create_model( 'mobilenetv2_100', pretrained=True, features_only=True, )

but it only works with intenet connection and pretrained being True. However the machine i work on doesnt have internet access, hence I decided to load it from local, but I dont know if there's a way to do so.

Can you please tell me how to load a pretrained model from local path?

[hankyul2]

Hi @ShallJaiden

(1) You can download the pre-trained weight and find the download path, usually the ~/.cache/torch/hub/checkpoints folder.
(2) You can manually load the downloaded pre-trained weight from the local path using the checkpoint_path option like the code snippet below.

import timm
model = timm.create_model('resnet50', checkpoint_path='/home/hankyul/.cache/torch/hub/checkpoints/resnet50_a1_0-14fe96d1.pth')

Thank you.

hankyul

[watertianyi]

你好@ShallJaiden

（1）您可以下载预先训练的权重并找到下载路径，通常是~/.cache/torch/hub/checkpoints文件夹。 （2）您可以使用类似下面代码片段的选项 从本地路径手动加载下载的预先训练的权重。checkpoint_path

import timm
model = timm.create_model('resnet50', checkpoint_path='/home/hankyul/.cache/torch/hub/checkpoints/resnet50_a1_0-14fe96d1.pth')

谢谢。

韩剧

https://github.com/huggingface/pytorch-image-models/issues/2061#issuecomment-2407012145，
'convnextv2_tiny.fcmae': _cfgv2(
url="https://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_tiny_1k_224_fcmae.pt",
hf_hub_id='timm/',
num_classes=0),

[rwightman]

You can also do this to load the weights as if they were the main pretrained weights (overriding the source from URL or HF hub -> local file), this lets things like features_only or head adaptation to diff # classes work ore easily

#1826 (comment)

[rwightman]

timm.create_model(
  'swinv2_large_window12to16_192to256',
  pretrained=True,
  pretrained_cfg_overlay=dict(file='path/to/checkpoint'),
)

ths makes the local file work exactly the same as the online pretrained weights for use, where as checkpoint_path= needs to load exactly the exact model config that's created

[abdelkareemkobo]

@rwightman How can i load a Dinov2 into timm?
I have fine-tuned the Dinov2 with HF Trainer and the final checkpoint is :

['scheduler.pt',
 'model.safetensors',
 'config.json',
 'rng_state.pth',
 'trainer_state.json',
 'training_args.bin',
 'optimizer.pt',
 'preprocessor_config.json']

model = timm.create_model(
    "vit_base_patch14_dinov2",
    pretrained=True,   pretrained_cfg_overlay=dict(file="/home/kareem/hacking/research/AI_Love/Dinov2_for_cancer/lung_cancer_Iq/dinov2-base-finetuned-ct-iq/checkpoint-270"),
)

I get FileNotFoundError:
Any help how to load Dinov2 with timm, thanks in advance

[rwightman]

@abdelkareemkobo you'd need to point directly to the model weights which is model.safetensors ... though, if this is a Transformers instance of Dinov2 it won't load into timm.

[watertianyi]

@rwightman #2061 (comment)
How to load a model you trained yourself？
Shouldn’t the above weights be the model imported from fcmae? How come the model imported from convnextv2 is imported?

[rwightman]

@goldwater668 timm convnexts are different layout than the original convnext / convnextv2 repo, the weights need to be remapped, that can be done by forcing them to be loaded as pretrained weights (mapping fn is applied), or you can run this mapping function manually on the state dict before loading https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/convnext.py#L509

[watertianyi]

@goldwater668timm convnexts 与原始 convnext / convnextv2 repo 的布局不同，需要重新映射权重，可以通过强制将它们加载为预训练权重（应用映射 fn）来完成，或者您可以在加载之前在状态字典上手动运行此映射函数https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/convnext.py#L509

@rwightman
Thank you for your reply. Yes, I have seen the weight filtering code, but why do I get an error when I pass in timm.create_model(model_name, pretrained=False, checkpoint_path=checkpoint_path)?
The second problem is that the weights I trained myself mismatch for stem.0.weight: copying a param with shape torch.Size([64, 4, 4, 4]) from checkpoint, the shape in current model is torch.Size ([64, 3, 4, 4]).

Original downsample_layers.0.0.weight torch.Size([64, 3, 4, 4])
Mine is downsample_layers.0.0.weight torch.Size([64, 4, 4, 4])

[rwightman]

If there's 4 input channels you need to create model with in_chans=4.

[watertianyi]

@rwightman
Both create_dataset and create_loader are for 3-channel or single-channel normal images. If I input four-channel data, how should I modify the parameters? Because my four-channel data is spliced ​​from four single-channel 2D images.