Way to relax constrain for prefix length

[inspirit]

I think we can get away with having learnable null-latents in case we dont have prefix initially

[lucidrains]

@inspirit but you need at least one token that outputs a logit

[lucidrains]

yeah, or maybe the other way is to randomly curtail the prefix during training, in which case it will generalize on being conditioned from 0 prefix length to the maximum

[lucidrains]

feel like the paper should have addressed this, especially if book-level autoregressive generation is the goal here

[inspirit]

i think having prefix and query dynamically sized is the best for robustness as well as inference usage

[lucidrains]

@inspirit yeah, maybe i'll just have to push this responsibility to the dataloading

[ArEnSc]

@lucidrains I searched in the paper I don't see a prefix length mentioned. I am confused about this prefix length issue, wouldn't you want the prefix length to be the full size of the context window? ( I guess you guys meant the length of the latents )