Skip to content

Latest commit

 

History

History
231 lines (211 loc) · 6.35 KB

File metadata and controls

231 lines (211 loc) · 6.35 KB

Results of GPT-3.5 on HumanEval/160

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 233

def do_algebra(operator, operand):
    """
    Given two lists operator, and operand. The first list has basic algebra operations, and 
    the second list is a list of integers. Use the two given lists to build the algebric 
    expression and return the evaluation of this expression.

    The basic algebra operations:
    Addition ( + ) 
    Subtraction ( - ) 
    Multiplication ( * ) 
    Floor division ( // ) 
    Exponentiation ( ** ) 

    Example:
    operator['+', '*', '-']
    array = [2, 3, 4, 5]
    result = 2 + 3 * 4 - 5
    => result = 9

    Note:
        The length of operator list is equal to the length of operand list minus one.
        Operand is a list of of non-negative integers.
        Operator list has at least one operator, and operand list has at least two operands.

    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def do_algebra(operator, operand):
    """
    Given two lists operator, and operand. The first list has basic algebra operations, and 
    the second list is a list of integers. Use the two given lists to build the algebric 
    expression and return the evaluation of this expression.

    The basic algebra operations:
    Addition ( + ) 
    Subtraction ( - ) 
    Multiplication ( * ) 
    Floor division ( // ) 
    Exponentiation ( ** ) 

    Example:
    operator['+', '*', '-']
    array = [2, 3, 4, 5]
    result = 2 + 3 * 4 - 5
    => result = 9

    Note:
        The length of operator list is equal to the length of operand list minus one.
        Operand is a list of of non-negative integers.
        Operator list has at least one operator, and operand list has at least two operands.

    """expression = str(operand[0])
    for oprt, oprn in zip(operator, operand[1:]):
        expression+= oprt + str(oprn)
    return eval(expression)

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate(['**', '*', '+'], [2, 3, 4, 5]) == 37
    assert candidate(['+', '*', '-'], [2, 3, 4, 5]) == 9
    assert candidate(['//', '*'], [7, 3, 4]) == 8, "This prints if this assert fails 1 (good for debugging!)"

    # Check some edge cases that are easy to work out by hand.
    assert True, "This prints if this assert fails 2 (also good for debugging!)"

Runs

✅ Run 1

Start time: 2023-07-18 17:01:28
Duration: 2.51s
Completion tokens: 48
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 2

Start time: 2023-07-18 17:01:25
Duration: 2.97s
Completion tokens: 48
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 3

Start time: 2023-07-18 17:01:22
Duration: 2.98s
Completion tokens: 47
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 4

Start time: 2023-07-18 17:01:20
Duration: 2.15s
Completion tokens: 47
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 5

Start time: 2023-07-18 17:01:18
Duration: 2.33s
Completion tokens: 48
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 6

Start time: 2023-07-18 17:01:08
Duration: 9.36s
Completion tokens: 238
Result: Pass

def do_algebra(operator, operand):
    """
    Given two lists operator, and operand. The first list has basic algebra operations, and 
    the second list is a list of integers. Use the two given lists to build the algebraic 
    expression and return the evaluation of this expression.

    The basic algebra operations:
    Addition ( + ) 
    Subtraction ( - ) 
    Multiplication ( * ) 
    Floor division ( // ) 
    Exponentiation ( ** ) 

    Example:
    operator['+', '*', '-']
    array = [2, 3, 4, 5]
    result = 2 + 3 * 4 - 5
    => result = 9

    Note:
        The length of operator list is equal to the length of operand list minus one.
        Operand is a list of non-negative integers.
        Operator list has at least one operator, and operand list has at least two operands.

    """
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 7

Start time: 2023-07-18 17:01:06
Duration: 2.19s
Completion tokens: 47
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 8

Start time: 2023-07-18 17:01:03
Duration: 2.54s
Completion tokens: 48
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 9

Start time: 2023-07-18 17:01:01
Duration: 2.62s
Completion tokens: 47
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)

✅ Run 10

Start time: 2023-07-18 17:00:58
Duration: 2.9s
Completion tokens: 47
Result: Pass

def do_algebra(operator, operand):
    expression = str(operand[0])
    for i in range(len(operator)):
        expression += operator[i] + str(operand[i+1])
    return eval(expression)