Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: Assertion failure: prealloc_size <= total_mem when I try to train the Lego #38

Open
Zdmai opened this issue Apr 19, 2023 · 2 comments

Comments

@Zdmai
Copy link

Zdmai commented Apr 19, 2023

RTX3050 mobile, Ubuntu 22.04.2 LTS x86_64

I don't know how to solve this problem, I try to change the patch_size into 2048 even 512 in train_nsvf_lego.sh and train.py but it can't work for me.

What can I do next.

Screenshot from 2023-04-20 00-36-17

./scripts/train_nsvf_lego.sh
[Taichi] version 1.7.0, llvm 15.0.4, commit 40ba02a3, linux, python 3.10.6
[Taichi] Starting on arch=cuda
[E 04/20/23 00:28:48.932 11717] [llvm_runtime_executor.cpp:materialize_runtime@570] Assertion failure: prealloc_size <= total_mem

Traceback (most recent call last):
File "/home/zzz/lec/taichi-nerfs/train.py", line 279, in
taichi_init(hparams)
File "/home/zzz/lec/taichi-nerfs/train.py", line 273, in taichi_init
ti.init(**taichi_init_args)
File "/home/zzz/.local/lib/python3.10/site-packages/taichi/lang/misc.py", line 455, in init
impl.get_runtime().prog.materialize_runtime()
RuntimeError: [llvm_runtime_executor.cpp:materialize_runtime@570] Assertion failure: prealloc_size <= total_mem

@bobcao3
Copy link

bobcao3 commented Apr 19, 2023

You would need to find the ti.init statement and then lower the memory allocation number.

@Zdmai
Copy link
Author

Zdmai commented Apr 20, 2023

yes! I solve this (last night, but my roommate have to sleep so I go to sleep too) by modify the

device_memory_GB to 1.0 that is work

image

but it have another error for me!

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants