Does modyn have a tutorial readme for DLRM online training?

[freshduer]

I watched it for a long time and still didn't understand how to run.。。

I have ran through the environment and the ./scripts/python_compliance.sh(pytest) in conda environment.

i do not have any docker env

for example, I want to run run_criteo_tput.py

how should I do.

best regards

[freshduer]

I watched it for a long time and still didn't understand how to run.。。

I have ran through the environment and the ./scripts/python_compliance.sh(pytest) in conda environment.

i do not have any docker env,but i have successfully run all components in my conda env

for example, I want to run run_criteo_tput.py

how should I do.

best regards

supervisor:

client:

[MaxiBoether]

Hey,

the file you linked is a throughput benchmark. I am quite sure you don't want to run this, unless you want to benchmark the throughput of the system. The errors you see indicate you don't have loaded the Criteo data. We cannot just share the preprocessed files with you due to copyright reasons. We provide preprocessing scripts at https://github.com/eth-easl/modyn/tree/main/benchmark/criteo_1TB for your convenience.

I can share this document with you: https://docs.google.com/document/d/1SHf2d5qIfbR1aXemd0MY31r_hJQvtn3MbSGIaDFqBRk/edit?usp=sharing

Note that this document is tailored towards our local machines in our group, so some steps like configuring rootless docker may be different for your setup. Step 2 is obtaining the data, which you need to do differently. You can also use a different dataset, instead of relying on Criteo. Starting at step 3 you find information helpful for running a criteo training pipeline.