How to set the reuse factor

[davidecarini]

Hello to everyone!

If i have this CNN, there is a way to find the right values to set the reuse factor for every layer ?
For example, i can use 1 as reuse factor for the first Conv2D layer but what is the value of the reuse factor of the second layer?

Layer (type) Output Shape Param #

convolution_layer_1 (Conv2D (None, 26, 26, 32) 320
)

ReLu_1 (Activation) (None, 26, 26, 32) 0

max_pooling_layer_1 (MaxPoo (None, 13, 13, 32) 0
ling2D)

convolution_layer_2 (Conv2D (None, 11, 11, 14) 4046
)

ReLu_2 (Activation) (None, 11, 11, 14) 0

max_pooling_layer_2 (MaxPoo (None, 5, 5, 14) 0
ling2D)

flatten_layer (Flatten) (None, 350) 0

dropout_layer (Dropout) (None, 350) 0

dense_layer (Dense) (None, 10) 3510

softmax (Activation) (None, 10) 0

=================================================================
Total params: 7,876
Trainable params: 7,876
Non-trainable params: 0

[bo3z]

In general, the reuse factor (RF) for every layer will depend on preferences, such as latency vs. resources utilised and target architecture. A RF equal to one will mean all multiplications are executed in parallel. So, in general, for a layer of n_mult multiplications and reuse factor of reuse_factor, we expect the layer to use n_mult / reuse_factor DSPs. This number can vary sometimes, due to various reasons - e.g. for low-precision (less than 9 bits), the HLS compiler will implement multiplication through combinatorial logic (LUT), rather than DSPs. The number of expected DSPs can also vary, depending on the DSP operating mode, target device etc. But, in general, this estimate is very accurate. Finally, there are some savings when your weight matrix is sparse and RF = 1 (however, when RF > 1, this is not the case)

For Dense layers, there is n_in * n_out multiplications, thus calculating the entire output after all the multiplications are done. For Conv2D layers there is n_filt * n_chan * filt_width * filt_height mutliplications for one output pixel (across all the output channels). By default, one output pixel is computed, before proceeding to the next one, but this could be tuned.

In general, increasing RF will mean splitting the multiplications across more clock cycles - RF = 1 is equivalent to all multiplications done in parallel and RF = 4 would mean the multiplications are done across four clock cycles, with (approx.) four times less DSPs (as well as a reduction in other resources, but this reduction is not linear). So for a Dense layer, the expected latency is equal to the RF (plus some overhead to retrieve the weights from BRAM, if using Resource strategy). For Conv2D layers, the expected latency is proportional to the product of output pixels (height x width) and the RF.

Depending on your target device, latency constraints etc. the RF should be set accordingly for every layer. In your case, the model is fairly small and very high-end FPGAs (~8k DSPs) could accomodate the entire model with RF = 1. However, if the goal is to save resources - it might be worth increasing RF.

Examples on changing reuse factor, precision etc, as well as a more in-depth explanation on RF and DSPs used, are available in the tutorial.

One possible approach is to generate the configuration (Python dict) from the model, given a default reuse factor and then overwrite it for individual layers:

default_reuse_factor = 2
hls_config = config_from_keras_model(baseline_model, default_reuse_factor=default_reuse_factor)
hls_config['LayerName']['convolution_layer_2']['ReuseFactor'] = 4

The above code will generate an hls4ml config (a Python dictionary with all the necessary information for model conversion) with a default reuse factor of 2 for every layer. Then, the reuse factor can be overwritten, by changing the config for specific layers in the dictionary - hls_config['LayerName'][layer_name]['ReuseFactor'] = new_reuse_factor. In the above example, default reuse factor for all layers is 2 and in the second Conv2D layer it is 4. If not specified, the default reuse factor is set to 1 (for all layers)

[bo3z]

It could simply be the server time-out - the lower the RF, the more scheduling the compiler needs to do. Again, depending on your server resources, target device etc. HLS synthesis can take very long, even several hours (3-4) is relatively common. It might be worth exploring ways of keeping the server connection running.

[davidecarini]

Ok, thus your answer is that the synthesis could work also with RF = 1 for all the layers and the problem is the server connection time out?

[bo3z]

Not sure if RF = 1 will fit on the board, but it is worth trying a longer time-out / running the process after disconnecting from the server. Given larger models (e.g. in the tutorial) succeeded synthesis with lower RF, it should certainly be possible. Alternatively, depending on your dataset, maybe reduce model complexity by reducing the number of filters in the first layer?

[coldCecemel]

Why is that your available reusefactors change depending on the chosen strategy? (resource or latency)

[bo3z]

ReuseFactor doesn't depend on the strategy, only on layer input and output shapes.