Help with loading model

[Axel-At-Apollo]

See #1410, has not been resolved as of yet.

[KareemMusleh]

I got your code to work like this

modeling_llama = importlib.reload(transformers.models.llama.modeling_llama)

model = modeling_llama.LlamaForCausalLM.from_pretrained(checkpoint_path)
tokenizer = transformers.AutoTokenizer.from_pretrained(checkpoint_path)

# Add text generation
prompt = "Write a short story about a robot learning to paint:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    inputs.input_ids,
    max_length=200,
    num_return_sequences=1,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id,
)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("\nGenerated text:")
print(generated_text)

I will try to find a more general solution