Skip to content

Latest commit

 

History

History
215 lines (201 loc) · 6.6 KB

101.md

File metadata and controls

215 lines (201 loc) · 6.6 KB

Results of GPT-3.5 on HumanEval/101

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 142

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """if not s:
        return []

    s_list = []

    for letter in s:
        if letter == ',':
            s_list.append(' ')
        else:
            s_list.append(letter)

    s_list = "".join(s_list)
    return s_list.split()

Unit tests

def check(candidate):

    # Check some simple cases
    assert True, "This prints if this assert fails 1 (good for debugging!)"
    assert candidate("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    assert candidate("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    assert candidate("Hi, my name") == ["Hi", "my", "name"]
    assert candidate("One,, two, three, four, five, six,") == ["One", "two", "three", "four", "five", "six"]

    # Check some edge cases that are easy to work out by hand.
    assert True, "This prints if this assert fails 2 (also good for debugging!)"
    assert candidate("") == []
    assert candidate("ahmed     , gamal") == ["ahmed", "gamal"]

Runs

✅ Run 1

Start time: 2023-07-18 17:12:41
Duration: 1.14s
Completion tokens: 14
Result: Pass

def words_string(s):
    return s.replace(",", " ").split()

✅ Run 2

Start time: 2023-07-18 17:12:36
Duration: 4.68s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 3

Start time: 2023-07-18 17:12:31
Duration: 4.64s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 4

Start time: 2023-07-18 17:12:30
Duration: 1.1s
Completion tokens: 14
Result: Pass

def words_string(s):
    return s.replace(",", " ").split()

✅ Run 5

Start time: 2023-07-18 17:12:25
Duration: 4.7s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 6

Start time: 2023-07-18 17:12:23
Duration: 1.82s
Completion tokens: 14
Result: Pass

def words_string(s):
    return s.replace(",", " ").split()

✅ Run 7

Start time: 2023-07-18 17:12:18
Duration: 5.13s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 8

Start time: 2023-07-18 17:12:13
Duration: 4.74s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 9

Start time: 2023-07-18 17:12:08
Duration: 4.93s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words

✅ Run 10

Start time: 2023-07-18 17:12:03
Duration: 5.01s
Completion tokens: 123
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    words = s.replace(",", " ").split()
    return words