NeMo CPU conversion

[mgilmore-relace]

Is your feature request related to a problem? Please describe.

Hello, I am trying to do a conversion of a very largo model. Qwen3-coder-480B. The conversion is only using the CPU, which is desired since the model cannot fit into a single GPU. The issue is that the GPU nodes are capped at ~1.5TB of RAM. This is enough to load the model and do the conversion, but fails when writing from running out of RAM.

Describe the solution you'd like

There are nodes with large ram 4TB+, but without GPUs. This fails because the megatron initialization requires a GPU. Is there a way the conversion can support CPU only systems?

Describe alternatives you've considered

Tried megatron-bridge, nemo llm import, and ms-swift conversion tool. All seem to have the same GPU requirement.

[decimal-agent]

This is an AI-generated response. Please verify.

To convert large models like Qwen3-coder-480B on CPU-only systems with large RAM, you'll need to modify the NeMo conversion process to bypass GPU requirements.

The key is to modify the model initialization code to skip GPU-specific initialization while still using the use_cpu_initialization=True parameter in TransformerConfig.

Create a custom conversion script based on megatron_export.py that:

1. Removes all CUDA device initialization code
2. Modifies the fake_initialize_model_parallel function to work without GPU
3. Forces all tensors to remain on CPU throughout the conversion process

This approach leverages your large RAM capacity (4TB+) while avoiding the GPU requirement that's causing the current failure when writing the converted model.