Add GPTQ Marlin support for 4 and 8 bit

[EricLBuehler]

Increased speed on CUDA is to be expected.

[github-actions]

Code Metrics Report

  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 C Header                2           35           28            0            7
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                   12          105          104            0            1
 Python                 52         2280         1940           68          272
 TOML                   20          630          564            2           64
 YAML                    2           21           19            2            0
-------------------------------------------------------------------------------
 Jupyter Notebooks       4            0            0            0            0
 |- Markdown             2           77           32           31           14
 |- Python               2          196          169            1           26
 (Total)                            273          201           32           40
-------------------------------------------------------------------------------
 Markdown               38         2800            0         2129          671
 |- BASH                 6          103          100            0            3
 |- JSON                 1           12           12            0            0
 |- Python               5           92           82            0           10
 |- Rust                 9          322          274            0           48
 |- TOML                 2           75           63            0           12
 (Total)                           3404          531         2129          744
-------------------------------------------------------------------------------
 Rust                  267        78791        70730         1671         6390
 |- Markdown           131         1363           25         1243           95
 (Total)                          80154        70755         2914         6485
===============================================================================
 Total                 400        85138        73779         3872         7487
===============================================================================
  

[EricLBuehler]

RUST_BACKTRACE=1 cargo run --features cuda -- -i plain -m kaitchup/Phi-3-mini-4k-instruct-gptq-4bit