Support in-situ conversion of GPTQ model to marlin format (naive GPTQ kernel also supported).

[guoqingbao]

For quantized 4-bit GPTQ model:

cargo run --release -- --port 2000 --weight-path /home/mistral_7b-int4/ mistral --quant marlin

It performs in-situ conversion of GPTQ model to marlin format during model loading.

Please note:

Marlin format in-situ conversion only support 4-bit GPTQ (with sym=True, groupsize=128 or -1, desc_act=False).