Profiling with PPE for Local

[kuenishi]

PR draft to discuss the base design for #258.

With a test script like below:

import pytorch_pfn_extras as ppe

from pfio.v2 import Local, Path

@ppe.profiler.record_function("my_tag_2")
def my_run2(x):
    pass

def my_run1(x):
    pass

def p():
    with ppe.profiler.record("my_tag_1"):
        my_run1('hoge')

    for i in range(10):
        my_run2("far")

    total = 0
    count = 0
            
    p = Path()
    for f in p.glob("*"):
        s = f.stat()
        total += s.size
        count += 1

    d = 0
    with Local() as fs:
        for f in fs.list():
            if fs.isdir(f.strip()):
                d += 1
                continue
                
            len(fs.open(f).read())

    print(count - d, "files", d, "dirs", total, "bytes for files")

    ts = ppe.profiler.get_time_summary()

    with ts.summary() as o:
        print(o)

p()

It yields a result like this:

$ python p.py
15 files 13 dirs 8490 bytes for files
(<pytorch_pfn_extras.reporting.DictSummary object at 0x7fae6b3204d0>, {'my_tag_1.min': 9.5367431640625e-06, 'my_tag_1
.max': 9.5367431640625e-06, 'my_tag_2.min': 9.5367431640625e-07, 'my_tag_2.max': 2.86102294921875e-06, 'local-stat.mi
n': 9.059906005859375e-06, 'local-stat.max': 4.9591064453125e-05, 'local-open.min': 1.3828277587890625e-05, 'local-op
en.max': 6.914138793945312e-05, 'local-list.min': 6.9141387939453125e-06, 'local-list.max': 0.00014781951904296875})

[kuenishi]

Latest:

$ python p.py
15 files 13 dirs 8491 bytes for files
(<pytorch_pfn_extras.reporting.DictSummary object at 0x7f5bbc7207d0>, {'my_tag_1.min': 1.0967254638671875e-05, 'my_ta
g_1.max': 1.0967254638671875e-05, 'my_tag_2.min': 7.152557373046875e-07, 'my_tag_2.max': 3.0994415283203125e-06, 'pfi
o.v2.Local:glob.min': 0.0004813671112060547, 'pfio.v2.Local:glob.max': 0.0004813671112060547, 'pfio.v2.Local:stat.min
': 8.106231689453125e-06, 'pfio.v2.Local:stat.max': 0.00011372566223144531, 'pfio.v2.Local:open.min': 1.3113021850585
938e-05, 'pfio.v2.Local:open.max': 6.628036499023438e-05, 'pfio.v2.Local:list.min': 8.344650268554688e-06, 'pfio.v2.L
ocal:list.max': 0.0001614093780517578})

[y1r]

Thank you for providing the base design. Overall, LGTM.

Several questions are following:

• Writing annotations in all FS seems complicated, is it possible to implement them in FS (interface) and then override them if necessary?
• How do you calculate the cache hit rate in this design? Count call rate of slow FS?
• How do you measure read/write performance?

[y1r]

This implementation should be in pfio/v2/_profiler.py or pfio/_profiler.py because these noop implementation will be used in all FS.

[kuenishi]

Superseded by #340 .