[FEATURE] Torchserve documentation/examples

[levan92]

Is your feature request related to a problem? Please describe.
No torchserve support or examples

Describe the solution you'd like
Torchserve conversion scripts/examples

Additional context
Any plans for torchserve support or (just in case i missed it) is it already documented somewhere?

[rwightman]

@levan92 should be no different than torchvision, https://github.com/pytorch/serve/tree/master/examples/image_classifier/resnet_18 ... but using timm.models.resnet18 (or timm.create_model('resnet18', pretrained=True), or ImageClassifier(timm.models.ResNet), etc...

[levan92]

Thanks for your prompt reply! Yup, I've tried it out, you're right, it's very close to torchserve's default image_classifier handler, especially if using the default settings.

However, if we are using non-default preprocessing (which are supported in timm, say, crop_mode="border") or non-default model settings, we may need to modify the handler script.

Overriding the initialize method of ImageClassifier object within the handler script, these lines can be added:

        self.manifest = context.manifest

        model_dir = properties.get("model_dir")
        self.model_pt_path = None
        if "serializedFile" in self.manifest["model"]:
            serialized_file = self.manifest["model"]["serializedFile"]
            self.model_pt_path = os.path.join(model_dir, serialized_file)

        model_json_path = os.path.join(model_dir, "model.json")
        with open(model_json_path,'r') as rf: 
            model_dict = json.load(rf)
        self.model = create_model(
            checkpoint_path=self.model_pt_path, **model_dict)  # from timm.models import create_model
        self.model.to(self.device)
        self.model.eval()

        logger.debug("Model loaded successfully")

        # Preprocessing transforms
        data_json_path = os.path.join(model_dir, "data.json")
        with open(data_json_path,'r') as rf: 
            data_dict = json.load(rf)
        self.image_processing = transforms_imagenet_eval(**data_dict)  # from timm.data.transforms_factory import transforms_imagenet_eval

    

Then various arguments for preprocessing transforms/ model creation can be given through the respective jsons.