Replace stride with dilation in ResNet

[NielsRogge]

In torchvision's implementation of Resnet, there is a replace_stride_with_dilation parameter at the init. It's a list of length 3 that can be set to [False, False, True] for example, if you want to replace strided convolution by dilated convolution in the last convolutional block of the model.

I wonder how can I achieve the same thing with timm's implementation? (Reason for this is I'd like to implement Facebook AI's DETR model with support for any timm backend). Facebook itself has 4 variants, two of which use regular ResNet50/101, and two of which replace stride by dilation in the last block.

[rwightman]

@NielsRogge for models that support it, there is an output_stride arg which is default 32 for almost every model whether it has the arg or not. For models that I've added dilation support to, you can set output_stride=16 or output_stride=8. All timm models deriving from the ResNet class in resnet.py support it. Efficientnet/mobilenet, xception, csp, byob, regnet based nets support it. And then there are some that were too much effort for the payoff to deal with the striding...

[NielsRogge]

@rwightman thanks, setting output_stride to 16 seems to give equivalent results to torchvision's dilation=True for the last block.

Another question, so DETR has a convolutional backbone, and only the last feature map is used for object detection, but not for panoptic segmentation. In the latter case, several intermediate layers are also used in the segmentation head on top of the Transformer.

The problem is that I need to initialize an nn.Conv2d layer after the convolutional backbone, which projects the channel dimension of the final feature map of the backbone to the hidden size of the Transformer.

I want to do something like this:

from timm import create_model
import torch.nn as nn

self.backbone = create_model(model_name, pretrained=True, features_only=True, **kwargs)
self.input_projection = nn.Conv2d(self.backbone.num_channels, config.d_model, kernel_size=1)

How can I set this self.backbone.num_channels? I see each model has a num_features attribute, but a FeatureListNet doesn't have this:

AttributeError: 'FeatureListNet' object has no attribute 'num_features'

Would it be possible to add a num_features attribute to FeatureListNet? Or is there another way to access the channel dimension of the final feature map of a model?

[NielsRogge]

Nvm, found it by looking at: https://github.com/rwightman/pytorch-image-models/blob/a2727c1bf78ba0d7b5727f5f95e37fb7f8866b1f/timm/models/features.py

I can use self.backbone.feature_info.channels(-1)