Skip to content

Rebase to latest vLLM #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
wangxiyuan merged 48 commits into from
Jan 21, 2025
Merged

Rebase to latest vLLM #3

wangxiyuan merged 48 commits into from
Jan 21, 2025

Conversation

@MengqingCao
Copy link

@MengqingCao MengqingCao commented Jan 20, 2025 •
edited by github-actions bot
Loading 

* quickfix: using Torch SPDA when platform is out of tree

* Merge branch pr#11324 into apply_plugin

youkaichao and others added 30 commits January 16, 2025 19:58
@youkaichao
Support torchrun and SPMD-style offline inference (vllm-project#12071)
bf53e0c 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[core] LLM.collective_rpc interface and RLHF example (vllm-project#12084
92e793d 
)

Signed-off-by: youkaichao <[email protected]>
@ywang96
[Bugfix] Fix max image feature size for Llava-one-vision (vllm-projec…
874f7c2 
…t#12104)

Signed-off-by: Roger Wang <[email protected]>
@varun-sundar-rabindranath
[misc] Add LoRA kernel micro benchmarks (vllm-project#11579)
5fd24ec
@Isotr0py
[Model] Add support for deepseek-vl2-tiny model (vllm-project#12068)
62b06ba 
Signed-off-by: Isotr0py <[email protected]>
@heheda12345
[Bugfix] Set enforce_eager automatically for mllama (vllm-project#12127)
d06e824 
Signed-off-by: Chen Zhang <[email protected]>
@KuntaiDu
[Bugfix] Fix a path bug in disaggregated prefill example script. (vll…
ebc73f2 
…m-project#12121)

Signed-off-by: Kuntai Du <[email protected]>
@jikunshang
[CI]add genai-perf benchmark in nightly benchmark (vllm-project#10704)
fead53b 
Signed-off-by: Kunshang Ji <[email protected]>
@terrytangyuan
[Doc] Add instructions on using Podman when SELinux is active (vllm-p…
1475847 
…roject#12136)

Signed-off-by: Yuan Tang <[email protected]>
@terrytangyuan
[Bugfix] Fix issues in CPU build Dockerfile (vllm-project#12135)
b8bfa46 
Signed-off-by: Yuan Tang <[email protected]>
@heheda12345
[BugFix] add more is not None check in VllmConfig.__post_init__ (vl…
d1adb9b 
…lm-project#12138)

Signed-off-by: Chen Zhang <[email protected]>
@Isotr0py
[Misc] Add deepseek_vl2 chat template (vllm-project#12143)
d75ab55 
Signed-off-by: Isotr0py <[email protected]>
@divakar-amd
[ROCm][MoE] moe tuning support for rocm (vllm-project#12049)
8027a72 
Signed-off-by: Divakar Verma <[email protected]>
@heheda12345 @comaniac
[V1] Move more control of kv cache initialization from model_executor…
69d765f 
… to EngineCore (vllm-project#11960)

Signed-off-by: Chen Zhang <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
@jeejeelee
[Misc][LoRA] Improve the readability of LoRA error messages (vllm-pro…
07934cc 
…ject#12102)

Signed-off-by: Jee Jee Li <[email protected]>
@bigPYJ1151
[CI/Build][CPU][Bugfix] Fix CPU CI (vllm-project#12150)
d4e6194 
Signed-off-by: jiang1.li <[email protected]>
@youkaichao
[core] allow callable in collective_rpc (vllm-project#12151)
87a0c07 
Signed-off-by: youkaichao <[email protected]>
@wallashss
[Bugfix] Fix score api for missing max_model_len validation (vllm-pro…
58fd57f 
…ject#12119)

Signed-off-by: Wallas Santos <[email protected]>
@jikunshang
[Bugfix] Mistral tokenizer encode accept list of str (vllm-project#12149
54cacf0 
)

Signed-off-by: Kunshang Ji <[email protected]>
@gshtras
[AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (vllm-proje…
b5b57e3 
…ct#12134)

Signed-off-by: Gregory Shtrasberg <[email protected]>
@youkaichao
[torch.compile] disable logging when cache is disabled (vllm-project#…
7b98a65 
…12043)

Signed-off-by: youkaichao <[email protected]>
@youkaichao
[misc] fix cross-node TP (vllm-project#12166)
2b83503 
Signed-off-by: youkaichao <[email protected]>
@hongxiayang
[AMD][CI/Build][Bugfix] use pytorch stale wheel (vllm-project#12172)
c09503d 
Signed-off-by: hongxyan <[email protected]>
@youkaichao
[core] further polish memory profiling (vllm-project#12126)
da02cb4 
Signed-off-by: youkaichao <[email protected]>
@russellb
[Docs] Fix broken link in SECURITY.md (vllm-project#12175)
813f249 
Signed-off-by: Russell Bryant <[email protected]>
@Isotr0py
[Model] Port deepseek-vl2 processor, remove dependency (vllm-project#…
02798ec 
…12169)

Signed-off-by: Isotr0py <[email protected]>
@youkaichao
[core] clean up executor class hierarchy between v1 and v0 (vllm-proj…
6d0e3d3 
…ect#12171)

Signed-off-by: youkaichao <[email protected]>
@ice-tong
[Misc] Support register quantization method out-of-tree (vllm-project…
32eb0da 
…#11969)
@simon-mo
[V1] Collect env var for usage stats (vllm-project#12115)
7a8a48d
@madamczyk-intel
[BUGFIX] Move scores to float32 in case of running xgrammar on cpu (v…
4e94951 
…llm-project#12152)

Signed-off-by: Michal Adamczyk <[email protected]>
DarkLight1337 and others added 16 commits January 18, 2025 19:16
@DarkLight1337
[Bugfix] Fix multi-modal processors for transformers 4.48 (vllm-proje…
630eb5b 
…ct#12187)
@youkaichao
[torch.compile] store inductor compiled Python file (vllm-project#12182)
e66faf4 
Signed-off-by: youkaichao <[email protected]>
@gujingit @ywang96
benchmark_serving support --served-model-name param (vllm-project#12109)
936db11 
Signed-off-by: zibai <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@Isotr0py
[Misc] Add BNB support to GLM4-V model (vllm-project#12184)
edaae19 
Signed-off-by: Isotr0py <[email protected]>
@ywang96 @imkero @DarkLight1337
[V1] Add V1 support of Qwen2-VL (vllm-project#12128)
81763c5 
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: imkero <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@MartinGleize
[Model] Support for fairseq2 Llama (vllm-project#11442)
bbe5f9d 
Signed-off-by: Martin Gleize <[email protected]>
Co-authored-by: mgleize user <[email protected]>
@ShangmingCai
[Bugfix] Fix num_heads value for simple connector when tp enabled (vl…
df450aa 
…lm-project#12074)

Signed-off-by: Shangming Cai <[email protected]>
@youkaichao
[torch.compile] fix sym_tensor_indices (vllm-project#12191)
51ef828 
Signed-off-by: youkaichao <[email protected]>
@hmellor
Move linting to pre-commit (vllm-project#11975)
3ea7b94 
Signed-off-by: Harry Mellor <[email protected]>
@terrytangyuan
[DOC] Fix typo in docstring and assert message (vllm-project#12194)
c5c0620 
Signed-off-by: Yuan Tang <[email protected]>
@terrytangyuan
[DOC] Add missing docstring in LLMEngine.add_request() (vllm-project#…
d264312 
…12195)

Signed-off-by: Yuan Tang <[email protected]>
@terrytangyuan
[Bugfix] Fix incorrect types in LayerwiseProfileResults (vllm-project…
0974c9b 
…#12196)

Signed-off-by: Yuan Tang <[email protected]>
@Isotr0py
[Model] Add Qwen2 PRM model support (vllm-project#12202)
8360979 
Signed-off-by: Isotr0py <[email protected]>
@DarkLight1337
[Core] Interface for accessing model from VllmRunner (vllm-project#…
59a0192 
…10353)

Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[misc] add placeholder format.sh (vllm-project#12206)
5c89a29 
Signed-off-by: youkaichao <[email protected]>
@DarkLight1337
[CI/Build] Remove dummy CI steps (vllm-project#12208)
4001ea1 
Signed-off-by: DarkLight1337 <[email protected]>
@github-actions
Copy link

github-actions bot commented Jan 20, 2025

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@MengqingCao
Merge NPU related pr to apply_plugin
bfa00ad 
    * quickfix: using Torch SPDA when platform is out of tree

    * Merge branch pr#11324 into apply_plugin

Signed-off-by: <[email protected]>
Signed-off-by: MengqingCao <[email protected]>
@wangxiyuan wangxiyuan merged this pull request into cosdt:apply_plugin Jan 21, 2025
2 checks passed
@MengqingCao MengqingCao deleted the apply_plugin0120 branch February 25, 2025 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.