Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty content array instead of empty text #459

Closed
dandv opened this issue Jul 5, 2024 · 3 comments
Closed

Empty content array instead of empty text #459

dandv opened this issue Jul 5, 2024 · 3 comments

Comments

@dandv
Copy link
Contributor

dandv commented Jul 5, 2024

I filed this a few months ago (#399) and it turns out it's not a Bedrock issue, and not limited to max_tokens 1.

When Claude doesn't return anything, the content array is empty. This breaks the assumption that the content array will contain an object with a text field. The correct output should be

[ { type: 'text', text: '' } ]

Repro:

import axios from 'axios';
import 'dotenv/config';

interface Message {
  role: string;
  content: string;
}

async function apiCallMessages(messages: Message[], model: string) {
  const sampleRequest = {
    messages,
    max_tokens: 4096,  // required, maximum
    model,
  };

  let parsed, response;

  response = await axios.post('https://api.anthropic.com/v1/messages', sampleRequest, {
    headers: {
      'x-api-key': process.env.ANTHROPIC_API_KEY as string,
      'anthropic-version': '2023-06-01',  // required by the Messages API, optional for Text Completions
      'anthropic-beta': 'messages-2023-12-15',  // required to use the Messages API while in beta
      'anthropic-sdk': `anthropic-sheets/0.8.1`,
    },
  }) as Record<string, string>;
  return response.data;
}

const prompt = 'Who is the best basketball player of all time? Please choose one specific player and be brief.';
let response = await apiCallMessages([
  { role: 'user', content: prompt },
  { role: 'assistant', content: 'Stephen' },
], 'claude-3-haiku-20240307');
console.log(response['content']);  // 'Curry...'

response = await apiCallMessages([
  { role: 'user', content: prompt },
  { role: 'assistant', content: 'Stephen Curry.' },
], 'claude-3-haiku-20240307');
console.log(response['content']);  // <-- [], which is an error
@rattrayalex
Copy link
Collaborator

Where is it documented that content should always be a non-empty array? This may be a bug in the documentation, or a misunderstanding of the API's contract.

Either way, this does not sound like a bug in the SDK, so I'm going to go ahead and close this issue.

@rattrayalex rattrayalex closed this as not planned Won't fix, can't repro, duplicate, stale Jul 6, 2024
@aaron-lerner
Copy link

When Claude doesn't return anything, the content array is empty. This breaks the assumption that the content array will contain an object with a text field. The correct output should be

[ { type: 'text', text: '' } ]

@rattrayalex is correct, that is not a valid assumption and this is the expected behavior for empty responses and matches the behavior on input where "content": [{"type": "text", "text": ""}] will give you an error.

Unrelated note:

'anthropic-beta': 'messages-2023-12-15', // required to use the Messages API while in beta

Messages API is now GA and you no longer need this header.

@dandv
Copy link
Contributor Author

dandv commented Jul 9, 2024

Thanks for the clarifications!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants