How to set the external mode

[sansi-zhang]

Hi everyone,

If I want to run my application using external mode, what changes do I need to make from the original mode (e.g. MVU and VVAU)

Thanks.

[fpjentzsch]

Hi,

you should be able to set mem_mode of select MVAU/VVAU nodes to external via the folding config. The ZynqBuild flow should then attach DMA IP cores to their weight stream interfaces in the block design. The FINN-generated PYNQ driver should also support this feature:

finn/src/finn/qnn-data/templates/driver/driver_base.py

Line 113 in b3bdff1

def load_external_weights(self):

However, I've not used this feature myself for a while. Hopefully someone else can chime in if I missed anything.

[sansi-zhang]

Hi,

you should be able to set mem_mode of select MVAU/VVAU nodes to external via the folding config. The ZynqBuild flow should then attach DMA IP cores to their weight stream interfaces in the block design. The FINN-generated PYNQ driver should also support this feature:

finn/src/finn/qnn-data/templates/driver/driver_base.py

Line 113 in b3bdff1

def load_external_weights(self):

However, I've not used this feature myself for a while. Hopefully someone else can chime in if I missed anything.

Thank you for your reply.

You mean that if I want to use the external mode, I need to extract the desired weights for the layer I want to set and save the.npy file. Does this automatically discard the weights written into the onnx model?

[sansi-zhang]

Hi,

you should be able to set mem_mode of select MVAU/VVAU nodes to external via the folding config. The ZynqBuild flow should then attach DMA IP cores to their weight stream interfaces in the block design. The FINN-generated PYNQ driver should also support this feature:

finn/src/finn/qnn-data/templates/driver/driver_base.py

Line 113 in b3bdff1

def load_external_weights(self):

However, I've not used this feature myself for a while. Hopefully someone else can chime in if I missed anything.

I'm sorry to bother you again.

I directly set the external mode when setting folding as you suggested, but it will report that "orig_weight_matrix" is NoneType.

By querying the error message, I tried to save the weight.npy format weight of the layer I needed to set external in a specific folder, and set "code_gen_dir_ipgen" as the corresponding folder.

However, it will affect the storage location and read location of other files are different. This causes an error that the file cannot be found.

[surajkarki66]

Hello @sansi-zhang

Have you successfully implmented mem_mode="external" feature of FINN, because I am getting some issues while implementing this. If so, would you please explain me here.

[surajkarki66]

What is the proper way to do this ?

[surajkarki66]

Hi @fpjentzsch

I hope this message finds you well. I am currently working on deploying various YOLO model versions on a ZCU104 FPGA using FINN+. Due to model sizes, I'm encountering BRAM constraints that prevent some models from fitting onto the FPGA.

I wanted to explore using external memory for weight storage, but I have concerns about potential AXI Lite interface limitations when implementing this approach. Could you please advise:

Is there a recommended method to configure external memory access for weights in FINN+ ?

Are there specific considerations or constraints regarding AXI Lite connections when using external memory mode (using lot of MVAU/VVAU nodes with external mem_mode) ?

I would greatly appreciate your insights on managing these resource constraints while maintaining reasonable performance.

Thank you in advance for your time and expertise.

Best regards,
Suraj Karki
Hochschule Bielefeld