Skip to content

Latest commit

 

History

History
257 lines (255 loc) · 5.17 KB

48.md

File metadata and controls

257 lines (255 loc) · 5.17 KB

Results of GPT-4 on HumanEval/48

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 97

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """for i in range(len(text)):
        if text[i] != text[len(text) - 1 - i]:
            return False
    return True

Unit tests

METADATA = {}


def check(candidate):
    assert candidate('') == True
    assert candidate('aba') == True
    assert candidate('aaaaa') == True
    assert candidate('zbcd') == False
    assert candidate('xywyx') == True
    assert candidate('xywyz') == False
    assert candidate('xywzx') == False

Runs

✅ Run 1

Start time: 2023-07-18 18:37:51
Duration: 7.23s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 2

Start time: 2023-07-18 18:37:43
Duration: 7.77s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 3

Start time: 2023-07-18 18:37:37
Duration: 5.77s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 4

Start time: 2023-07-18 18:37:31
Duration: 6.39s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 5

Start time: 2023-07-18 18:37:25
Duration: 5.62s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 6

Start time: 2023-07-18 18:37:18
Duration: 6.41s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 7

Start time: 2023-07-18 18:37:12
Duration: 6.8s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 8

Start time: 2023-07-18 18:37:05
Duration: 6.68s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 9

Start time: 2023-07-18 18:36:59
Duration: 5.85s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]

✅ Run 10

Start time: 2023-07-18 18:36:50
Duration: 8.84s
Completion tokens: 77
Result: Pass

def is_palindrome(text: str):
    """
    Checks if given string is a palindrome
    >>> is_palindrome('')
    True
    >>> is_palindrome('aba')
    True
    >>> is_palindrome('aaaaa')
    True
    >>> is_palindrome('zbcd')
    False
    """
    return text == text[::-1]