Why I get mask on cpu and self on cuda error

[lucaslulucaslu]

Hi,

I am using VertexAI workbench jupyter notebook to run very simple codes with outlines like this:

import outlines

from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_NAME = "Qwen/Qwen3-8B"
 
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto")

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

outline_model = outlines.from_transformers(model,tokenizer) 

I got error message when I try to do inference with this outline_model

it says expect self and mask to be on the same device, but got mask on cup and self on cuda:0

Does anyone know how to solve this issue?

Thanks!

[lucaslulucaslu]

Go back to 1.1.1 version solve the issue, I think it’s a issue with new version