18.md

Results of GPT-3.5 on HumanEval/18

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 114

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlaping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlaping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """times = 0

    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            times += 1

    return times

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate('', 'x') == 0
    assert candidate('xyxyxyx', 'x') == 4
    assert candidate('cacacacac', 'cac') == 4
    assert candidate('john doe', 'john') == 1

Runs

✅ Run 1

Start time: 2023-07-18 17:27:21
Duration: 2.98s
Completion tokens: 73
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    count = 0
    if len(substring) == 0:
        return count
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 2

Start time: 2023-07-18 17:27:18
Duration: 3.55s
Completion tokens: 73
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    count = 0
    if len(substring) == 0:
        return count
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 3

Start time: 2023-07-18 17:27:12
Duration: 5.56s
Completion tokens: 136
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if substring == '':
        return 0
    for i in range(len(string) - len(substring) + 1):
        if string[i:i + len(substring)] == substring:
            count += 1
    return count

✅ Run 4

Start time: 2023-07-18 17:27:07
Duration: 4.76s
Completion tokens: 140
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if len(substring) == 0:
        return 0
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 5

Start time: 2023-07-18 17:27:02
Duration: 5.27s
Completion tokens: 134
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if substring == '':
        return count
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 6

Start time: 2023-07-18 17:26:57
Duration: 4.99s
Completion tokens: 135
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if substring == '':
        return 0
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 7

Start time: 2023-07-18 17:26:52
Duration: 5.15s
Completion tokens: 134
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if substring == '':
        return count
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 8

Start time: 2023-07-18 17:26:48
Duration: 3.36s
Completion tokens: 74
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    count = 0
    if len(substring) == 0:
        return 0
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 9

Start time: 2023-07-18 17:26:43
Duration: 5.52s
Completion tokens: 140
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    """ Find how many times a given substring can be found in the original string. Count overlapping cases.
    >>> how_many_times('', 'a')
    0
    >>> how_many_times('aaa', 'a')
    3
    >>> how_many_times('aaaa', 'aa')
    3
    """
    count = 0
    if len(substring) == 0:
        return 0
    for i in range(len(string) - len(substring) + 1):
        if string[i:i+len(substring)] == substring:
            count += 1
    return count

✅ Run 10

Start time: 2023-07-18 17:26:39
Duration: 3.34s
Completion tokens: 69
Result: Pass

def how_many_times(string: str, substring: str) -> int:
    count = 0
    if substring == '':
        return count
    for i in range(len(string) - len(substring) + 1):
        if string[i:i + len(substring)] == substring:
            count += 1
    return count