Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArrayFire Exception (Device out of memory:101) while decoding using Beam search decoder #1006

Open
abramovi opened this issue Apr 6, 2022 · 1 comment
Labels

Comments

@abramovi
Copy link

abramovi commented Apr 6, 2022

Bug Description

I am trying to run inference on a 30 min audio file using Beam search but it fail immediately with the following error:
terminate called after throwing an instance of 'af::exception' what(): ArrayFire Exception (Device out of memory:101): In function af_err af_matmul(void**, af_array, af_array, af_mat_prop, af_mat_prop) In file src/api/c/blas.cpp:237

Reproduction Steps

[Steps to reproduce. Please include, if needed for your issue:
This is the command I run:
!../../build/bin/asr/fl_asr_tutorial_inference_ctc \ --am_path=am_transformer_ctc_stride3_letters_300Mparams.bin \ --tokens_path=tokens.txt \ --lexicon_path=lexicon.txt \ --lm_path=lm_common_crawl_small_4gram_prun0-6-15_200kvocab.bin \ --logtostderr=true \ --sample_rate=16000 \ --beam_size=50 \ --beam_size_token=30 \ --beam_threshold=100 \ --lm_weight=1.5 \ --word_score=0 \ --audio_list=audio.lst

this is the output:
`I0406 17:32:38.839179 11262 InferenceCTC.cpp:66] Gflags after parsing
--flagfile=;--fromenv=;--tryfromenv=;--undefok=;--tab_completion_columns=80;--tab_completion_word=;--help=false;--helpfull=false;--helpmatch=;--helpon=;--helppackage=false;--helpshort=false;--helpxml=false;--version=false;--am_path=am_transformer_ctc_stride3_letters_300Mparams.bin;--audio_list=audio.lst;--beam_size=50;--beam_size_token=30;--beam_threshold=100;--lexicon_path=lexicon.txt;--lm_path=lm_common_crawl_small_4gram_prun0-6-15_200kvocab.bin;--lm_weight=1.5;--sample_rate=16000;--tokens_path=tokens.txt;--word_score=0;--alsologtoemail=;--alsologtostderr=false;--colorlogtostderr=false;--drop_log_memory=true;--log_backtrace_at=;--log_dir=;--log_link=;--log_prefix=true;--logbuflevel=0;--logbufsecs=30;--logemaillevel=999;--logfile_mode=436;--logmailer=/bin/mail;--logtostderr=true;--max_log_size=1800;--minloglevel=0;--stderrthreshold=2;--stop_logging_if_full_disk=false;--symbolize_stacktrace=true;--v=0;--vmodule=;

I0406 17:32:38.839272 11262 InferenceCTC.cpp:89] [Inference tutorial for CTC] Reading acoustic model from am_transformer_ctc_stride3_letters_300Mparams.bin
I0406 17:32:40.061904 11262 InferenceCTC.cpp:140] [Inference tutorial for CTC] Network is loaded.
I0406 17:32:40.433898 11262 InferenceCTC.cpp:152] [Inference tutorial for CTC] Number of classes/tokens in the network: 29
I0406 17:32:40.433930 11262 InferenceCTC.cpp:155] [Inference tutorial for CTC] Number of words in the lexicon: 200001
I0406 17:32:40.667213 11262 InferenceCTC.cpp:166] [Inference tutorial for CTC] Language model is constructed.
I0406 17:32:41.800864 11262 InferenceCTC.cpp:177] [Inference tutorial for CTC] Trie is planted.
I0406 17:32:41.801892 11262 InferenceCTC.cpp:196] [Inference tutorial for CTC] Beam search decoder is created

terminate called after throwing an instance of 'af::exception'
what(): ArrayFire Exception (Device out of memory:101):
In function af_err af_matmul(void**, af_array, af_array, af_mat_prop, af_mat_prop)
In file src/api/c/blas.cpp:237`

Platform and Hardware

[Please list your operating system, [GPU] hardware, compiler, and other details if relevant]
I am running on CPU machine with 4 core and 8 gb ram

@abramovi abramovi added the bug label Apr 6, 2022
@Manni-Arora
Copy link

Was getting the same error. Try reducing the size of your audio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants