79113992

Date: 2024-10-22 12:19:29
Score: 3
Natty:
Report link

This is a bug in the _load_pretrained_model() function of transformers/modeling_utils.py when loading sharded weight files. The state_dict is applied to the empty model per shard. This is problematic as the quantized weight and its meta data(*.quant_state.bitsandbytes__nf4) may be stored in the different shards. The quick-and-dirty fix is to merge tensors from all shards into one state_dict. Similar issues have been reported on the unsloth github issue 638

Reasons:
  • RegEx Blacklisted phrase (1): Similar issue
  • Contains signature (1):
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: schnell18