Considering the case empty request list is given to base model

[sadra-barikbin]

Hi there!

To consider the case in which an empty list is given to the base model. This might be needed at least because LightevalTask.construct_requests might produce a dict with empty entries.

[sadra-barikbin]

@clefourrier @NathanHB
There seems to be something wrong with nanotron 0.4.0 or how we import things from it. Running mere

from nanotron.logging import get_logger

located in hierarchical_logger.py raises import error:

from lighteval.logging.hierarchical_logger import hlog
src/lighteval/logging/hierarchical_logger.py:33: in <module>
    from nanotron.logging import get_logger
../../.local/lib/python3.10/site-packages/nanotron/logging.py:38: in <module>
    from nanotron.config.config import LoggingArgs
../../.local/lib/python3.10/site-packages/nanotron/config/__init__.py:2: in <module>
    from nanotron.config.config import *
../../.local/lib/python3.10/site-packages/nanotron/config/config.py:13: in <module>
    from nanotron.config.lighteval_config import LightEvalConfig
../../.local/lib/python3.10/site-packages/nanotron/config/lighteval_config.py:5: in <module>
    from nanotron.config.parallelism_config import ParallelismArgs
../../.local/lib/python3.10/site-packages/nanotron/config/parallelism_config.py:4: in <module>
    from nanotron.config.utils_config import (
../../.local/lib/python3.10/site-packages/nanotron/config/utils_config.py:8: in <module>
    from nanotron.parallel.pipeline_parallel.engine import (
../../.local/lib/python3.10/site-packages/nanotron/parallel/pipeline_parallel/__init__.py:1: in <module>
    from nanotron.parallel.pipeline_parallel.engine import PipelineEngine
../../.local/lib/python3.10/site-packages/nanotron/parallel/pipeline_parallel/engine.py:8: in <module>
    from nanotron.logging import log_rank
E   ImportError: cannot import name 'log_rank' from partially initialized module 'nanotron.logging' (most likely due to a circular import) (/home/sadrodin/.local/lib/python3.10/site-packages/nanotron/logging.py)

[NathanHB]

Hi ! Yes this is an issue on nanotron's side, they have been made aware

[sadra-barikbin]

@NathanHB , a test raises permission error when EnvConfig.cache param to AutoConfig.from_pretrained() (during loading BaseModel) has its default value /scratch. Has been choosing this path on purpose?

[sadra-barikbin]

@NathanHB, isn't better that EnvConfig's cache default value be huggingface_hub.constants.HF_HUB_CACHE?