Does hls4ml support reduce_ mean[tensorflow] operator---[begginer]

[zyt1024]

Hello, I am a beginner in hls4ml, and below is a test model. I want to know if reduce_mean in the following code is supported and what I should do to make it work？

from tensorflow.keras.layers import Input
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.regularizers import l1
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense

from tensorflow.keras.models import Model
import tensorflow as tf
import copy
filters_per_conv_layer = [128,128,128]
neurons_per_dense_layer = [1,32]
input_shape = (16, 128, 1)
input_shape_2 = (128, 128)
n_classes = 2
x = x_in = Input(input_shape)
fts = x_in2 = Input(input_shape_2)
for i,f in enumerate(filters_per_conv_layer):
    print( ('Adding convolutional block {} with N={} filters').format(i,f) )
    x = Conv2D(int(f), kernel_size=(1,1), strides=(1,1), kernel_initializer='lecun_uniform', kernel_regularizer=l1(0.0001), use_bias=True,
       name='conv_{}'.format(i))(x) 
    x = BatchNormalization(name='bn_conv_{}'.format(i))(x)
    x = Activation('relu',name='conv_act_%i'%i)(x)
print(tf.shape(x))
x = tf.reduce_mean(x, axis=1) 
conv1d = tf.keras.layers.Conv1D(int(128), kernel_size=1, strides=1,  kernel_initializer='lecun_uniform', kernel_regularizer=l1(0.0001),
                                use_bias=True,name="shot_conv")(fts)
x = Activation('relu',name='shot_act_1')(x + conv1d)

x = Flatten()(x)

for i,n in enumerate(neurons_per_dense_layer):
    print( ('Adding dense block {} with N={} neurons').format(i,n) )
    x = Dense(n,kernel_initializer='lecun_uniform', kernel_regularizer=l1(0.0001),name='dense_%i'%i, use_bias=True)(x)
    x = BatchNormalization(name='bn_dense_{}'.format(i))(x)
    x = Activation('relu',name='dense_act_%i'%i)(x)
x = Dense(int(n_classes),name='output_dense')(x)
x_out = Activation('softmax',name='output_softmax')(x)

model = Model(inputs=[x_in,x_in2], outputs=[x_out], name='keras_baseline')

model.summary()

[thaarres]

Hi @zyt1024! You can try
x = KL.GlobalMaxPooling1D(keepdims=True)(x)
as a supported alternative!

[jmduarte]

hm, yes I see the problem. GlobalMaxPooling1D does a mean over 1 axis, but it only accepts 3D tensors. GlobalMaxPooling2D accepts 4D tensors, but doesn't do the mean over just 1 axis (it does the mean over 2 axes). For now, you can try:

x = tf.keras.layers.Reshape((16, 128 * 128))(x)
x = tf.keras.layers.GlobalMaxPooling1D()(x)
x = tf.keras.layers.Reshape((128, 128))(x)

but a better way to do it may be to have a custom layer/implementation

[zyt1024]

When I tried using:

x = tf.keras.layers.Reshape((16, 128 * 128))(x)
x = tf.keras.layers.GlobalMaxPooling1D(x)
x = tf.keras.layers.Reshape((128, 128))(x)

I encountered a new error:

        26 x = tf.keras.layers.Reshape((16, 128 * 128))(x)
---> 27 x = tf.keras.layers.GlobalMaxPooling1D(x)
     28 x = tf.keras.layers.Reshape((128, 128))(x)
     29 x = Flatten()(x)
    221 if value is None:
    222     value = backend.image_data_format()
--> 223 data_format = value.lower()
    224 if data_format not in {"channels_first", "channels_last"}:
    225     raise ValueError(
    226         "The `data_format` argument must be one of "
    227         f'"channels_first", "channels_last". Received: {value}'
    228     )

AttributeError: 'KerasTensor' object has no attribute 'lower'

[jmduarte]

Sorry, there was a typo that I corrected. That line should be:

x = tf.keras.layers.GlobalMaxPooling1D()(x)

[zyt1024]

Thanks,now It's Ok,but BRAM resources are expensive.

x = tf.keras.layers.Reshape((16, 128 * 7))(x)
x = tf.keras.layers.GlobalMaxPooling1D()(x)
x = tf.keras.layers.Reshape((128, 7))(x)

    hls::stream<layer24_t> layer24_out("layer24_out");
    #pragma HLS STREAM variable=layer24_out depth=512
    nnet::repack_stream<layer10_t, layer24_t, 14336>(layer10_out, layer24_out); // repack_reshape_20

    hls::stream<layer12_t> layer12_out("layer12_out");
    #pragma HLS STREAM variable=layer12_out depth=1
    nnet::global_pooling1d_cl<layer24_t, layer12_t, config12>(layer24_out, layer12_out); // global_max_pooling1d_16
    hls::stream<layer25_t> layer25_out("layer25_out");
    #pragma HLS STREAM variable=layer25_out depth=128
    nnet::repack_stream<layer12_t, layer25_t, 896>(layer12_out, layer25_out); // repack_reshape_21

    |layer10_out_U  |        7|  161|   0|    -|   512|    112|    57344|
    |layer12_out_U  |        0|   99|   0|    -|     1|  14336|    14336|
    |layer15_out_U  |        0|   99|   0|    -|     1|     16|       16|
    |layer17_out_U  |        0|   99|   0|    -|     1|     16|       16|
    |layer18_out_U  |        0|   99|   0|    -|     1|     32|       32|
    |layer20_out_U  |        0|   99|   0|    -|     1|     32|       32|
    |layer21_out_U  |        0|   99|   0|    -|     1|     32|       32|
    |layer24_out_U  |      797|  161|   0|    -|   512|  14336|  7340032|
    |layer25_out_U  |        7|  157|   0|    -|   128|    112|    14336|
    |layer26_out_U  |      228|  161|   0|    -|   512|   4096|  2097152|
    |layer27_out_U  |      228|  161|   0|    -|   512|   4096|  2097152|

This layer24_ out occupies 797 BRAMs, which is too much,How to reduce BRAM?

[jmduarte]

Yes, the reshapes are (unnecessarily) expensive. At this point, what I would do is write a (very simple) custom layer to implement exactly what you want (you can call it a GlobalAveragePooling1DGeneric) following this example:

https://fastmachinelearning.org/hls4ml/advanced/extension.html

You may not even really need new HLS code to implement it.