Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

inference_prune.py比inference.py显存多 #2

Open
wuzuowuyou opened this issue Dec 17, 2020 · 25 comments
Open

inference_prune.py比inference.py显存多 #2

wuzuowuyou opened this issue Dec 17, 2020 · 25 comments

Comments

@wuzuowuyou
Copy link

inference_prune.py比inference.py显存多？多出400多M，这是为啥呢？
@wuzuowuyou
Copy link
Author

prune之后和原本的比较，显存没有减少，这是为啥呢？

@csliuchang
Copy link

大哥为啥我的inference没有编译成功，可以请教一下吗

@wuzuowuyou
Copy link
Author

贴上错误啊

@csliuchang
Copy link

Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found
In file included from include/pybind11/pytypes.h:12:0,
from include/pybind11/cast.h:13,
from include/pybind11/attr.h:13,
from include/pybind11/pybind11.h:43,
from adaptor.cpp:1:
include/pybind11/detail/common.h:112:30: fatal error: python3.5/Python.h: 没有那个文件或目录
#include <python3.5/Python.h>
^
compilation terminated.
make: *** [adaptor.so] Error 1
Traceback (most recent call last):
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/inference.py", line 23, in
from pse import pse
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/pse/init.py", line 9, in
raise RuntimeError('Cannot compile pse: {}'.format(BASE_DIR))
RuntimeError: Cannot compile pse: /home/liuchang/PycharmProjects/pse-lite.pytorch/pse
Makefile:10: recipe for target 'adaptor.so' failed
make: 离开目录“/home/liuchang/PycharmProjects/pse-lite.pytorch/pse”

Process finished with exit code 1

@csliuchang
Copy link

我用的python3.6

@wuzuowuyou
Copy link
Author

你这个是编译那个c++库不对啊，环境配置不对，改makefile

@fxwfzsxyq
Copy link

Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found
In file included from include/pybind11/pytypes.h:12:0,
from include/pybind11/cast.h:13,
from include/pybind11/attr.h:13,
from include/pybind11/pybind11.h:43,
from adaptor.cpp:1:
include/pybind11/detail/common.h:112:30: fatal error: python3.5/Python.h: 没有那个文件或目录
#include <python3.5/Python.h>
^
compilation terminated.
make: *** [adaptor.so] Error 1
Traceback (most recent call last):
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/inference.py", line 23, in
from pse import pse
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/pse/init.py", line 9, in
raise RuntimeError('Cannot compile pse: {}'.format(BASE_DIR))
RuntimeError: Cannot compile pse: /home/liuchang/PycharmProjects/pse-lite.pytorch/pse
Makefile:10: recipe for target 'adaptor.so' failed
make: 离开目录“/home/liuchang/PycharmProjects/pse-lite.pytorch/pse”

Process finished with exit code 1

这个https://github.com/BADBADBADBOY/pse-lite.pytorch/blob/master/pse/include/pybind11/detail/common.h 112-114行改成3.6

@csliuchang
Copy link

对的，改完之后少了一个bug，但是好像还是要重写makefile，不太会cpp编译。。。

@fxwfzsxyq
Copy link

对的，改完之后少了一个bug，但是好像还是要重写makefile，不太会cpp编译。。。

改完还不能编译成功？贴报错

@csliuchang
Copy link

Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found
adaptor.cpp:9:30: fatal error: opencv2/opencv.hpp: 没有那个文件或目录
#include <opencv2/opencv.hpp>
^
compilation terminated.
make: *** [adaptor.so] Error 1
Traceback (most recent call last):
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/inference.py", line 23, in
from pse import pse
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/pse/init.py", line 9, in
raise RuntimeError('Cannot compile pse: {}'.format(BASE_DIR))
RuntimeError: Cannot compile pse: /home/liuchang/PycharmProjects/pse-lite.pytorch/pse

@fxwfzsxyq
Copy link

Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found
adaptor.cpp:9:30: fatal error: opencv2/opencv.hpp: 没有那个文件或目录
#include <opencv2/opencv.hpp>
^
compilation terminated.
make: *** [adaptor.so] Error 1
Traceback (most recent call last):
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/inference.py", line 23, in
from pse import pse
File "/home/liuchang/PycharmProjects/pse-lite.pytorch/pse/init.py", line 9, in
raise RuntimeError('Cannot compile pse: {}'.format(BASE_DIR))
RuntimeError: Cannot compile pse: /home/liuchang/PycharmProjects/pse-lite.pytorch/pse

你装opencv了吗，要3.0+

@wuzuowuyou
Copy link
Author

给你贴一个我修改好的makfile，供你参考

CXXFLAGS = -I include  -std=c++11 -O3

DEPS = lanms.h $(shell find include -xtype f)
CXX_SOURCES = adaptor.cpp include/clipper/clipper.cpp
#OPENCV = `pkg-config --cflags --libs opencv`

INCLUDES = -I/data_1/Yang/software_install/Anaconda1105/envs/libtorch1.0/include
LIBDIRS = -L/data_1/Yang/software_install/Anaconda1105/envs/libtorch1.0/lib
LIBS = -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml

LIB_SO = adaptor.so

$(LIB_SO): $(CXX_SOURCES) $(DEPS)
	$(CXX) -o $@ $(CXXFLAGS) $(LDFLAGS) $(CXX_SOURCES) --shared -fPIC $(INCLUDES) $(LIBDIRS) $(LIBS)

clean:
	rm -rf $(LIB_SO)

@csliuchang
Copy link

应该是未配置c++版本的opencv，相关目录里面没有opencv文件

@wuzuowuyou
Copy link
Author

你们能测一下显存吗？我这里剪枝之后显存反而比之前多了，模型大小和参数量确实是减少了

@fxwfzsxyq
Copy link

你们能测一下显存吗？我这里剪枝之后显存反而比之前多了，模型大小和参数量确实是减少了

确实显存更大，我测试了，模型初始化的时候，确实是压缩的模型使用的显存更少一些，说明权重确实减少了，但是在模型forward的时候，同样大小的图片，反而是权重小的模型占用的显存更大，导致整体显存大于压缩后模型显存，我也很莫名其妙

@fxwfzsxyq
Copy link

你们能测一下显存吗？我这里剪枝之后显存反而比之前多了，模型大小和参数量确实是减少了

确实显存更大，我测试了，模型初始化的时候，确实是压缩的模型使用的显存更少一些，说明权重确实减少了，但是在模型forward的时候，同样大小的图片，反而是权重小的模型占用的显存更大，导致整体显存大于压缩后模型显存，我也很莫名其妙

如果在inference去掉with torch.no_grad() ，测试下来，又是压缩后的模型占用的显存小。可能问题出在这里

@wuzuowuyou
Copy link
Author

你们能测一下显存吗？我这里剪枝之后显存反而比之前多了，模型大小和参数量确实是减少了

确实显存更大，我测试了，模型初始化的时候，确实是压缩的模型使用的显存更少一些，说明权重确实减少了，但是在模型forward的时候，同样大小的图片，反而是权重小的模型占用的显存更大，导致整体显存大于压缩后模型显存，我也很莫名其妙

如果在inference去掉with torch.no_grad() ，测试下来，又是压缩后的模型占用的显存小。可能问题出在这里

为什么会这样，测试的时候就是应该加with torch.no_grad() 这个啊。
那你显存会小多少？？？

@fxwfzsxyq
Copy link

你们能测一下显存吗？我这里剪枝之后显存反而比之前多了，模型大小和参数量确实是减少了

确实显存更大，我测试了，模型初始化的时候，确实是压缩的模型使用的显存更少一些，说明权重确实减少了，但是在模型forward的时候，同样大小的图片，反而是权重小的模型占用的显存更大，导致整体显存大于压缩后模型显存，我也很莫名其妙

如果在inference去掉with torch.no_grad() ，测试下来，又是压缩后的模型占用的显存小。可能问题出在这里

为什么会这样，测试的时候就是应该加with torch.no_grad() 这个啊。
那你显存会小多少？？？

压缩模型：未压缩（压缩比不同可能不一样，这里只是我的测试结果）

初始化：
1264:1448

整体：
4020:3918

去掉with torch.no_grad():
14724:15918

@fxwfzsxyq
Copy link

解决掉这个问题应该就可以节省显存

@wuzuowuyou
Copy link
Author

解决掉这个问题应该就可以节省显存

去掉with torch.no_grad():
14724:15918
这个也没有少多少啊！

我模型每压缩之前是114M，压缩之后是51M，这个少了一半呢。
只是显存居然多了！！ 百思不得其解

@wuzuowuyou
Copy link
Author

__

@fxwfzsxyq
Copy link

__

我根据剪枝后的channnel，重新撸了一遍网络，在加了torch.no_grad的时候，发现就是这个网络计算的时候占得显存大，即便他少了一半的参数，放弃了,估计是pytorch的内部什么鬼机制

@wuzuowuyou
Copy link
Author

我觉得最大的可能就是剪枝之后的通道数不是2的次方数

@fxwfzsxyq
Copy link

fxwfzsxyq commented Dec 25, 2020 via email

@yu868
Copy link

yu868 commented Jul 28, 2022

关于这个问题大哥你后面有想到好的解决办法了吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fxwfzsxyq @wuzuowuyou @csliuchang @yu868