How the grad flow back while training APN?

[JohnnieXDU]

Hi, im trying to reimplement RACNN, and your src code helps me a lot.

But here are some questions confuse me.

1. How the grad flow back to APN while training APN? I mean that APN have 2 inputs: last image and coordinates[tx, ty, tl] from FC layer, and the output is cropped finer image. But while backprop, how to compute grad of [tx, ty, tl]?

2. Rank loss indeed takes as inputs two probabilities in paper, how to optimize 3 APN networks from 2 loss?

Thanks a lot.

[ouceduxzk]

1. this is nota a fully end-2-end training process, basically , you need to manually save the tx, ty, t from the highest response value of conv5_4 and you use this as the ground truth for training apn, you do not need to take care of gradient update at all
2. the idea is that net1 and net2 both gives a score s1 and s2, you use the rank that s2 > s1 to guide the net to learn better in the finer image

[zhangjian0826]

@ouceduxzk Hi, thanks for your code, but how to save the tx, ty, tl from the highest response value of conv5_4 ? Could you show me the code or just the steps？

[ouceduxzk]

@zhangjian0826 I did not have the code now, but the idea is that for each image, you do the inference and save the output of the conv5_4 and then you do some calculation on top of this output to find out the x, y coord of the max value

[jackshaw]

@ouceduxzk ,Hi,how do you think should we caculate the coordinate of the max response of conv5_4 ?I sum the feature maps along 512 channels to a single feature map and then find the location of the max response of this single feature map.