Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bedrock: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row error after the silent change to combine consecutive turns #565

Open
dandv opened this issue Oct 19, 2024 · 3 comments

Comments

@dandv
Copy link
Contributor

dandv commented Oct 19, 2024

The Messages API docs now state,

Consecutive user or assistant turns in your request will be combined into a single turn.

This change does not seem to be active for Bedrock yet. The code below returns a ValidationException:http://internal.amazon.com/coral/com.amazon.bedrock/:

await client.messages.create({
  max_tokens: 1,
  stream: false,
  messages: [
    { role: 'user', content: 'What is the color of' },
    { role: 'user', content: 'the sky?' },
  ],
  model: 'anthropic.claude-3-sonnet-20240229-v1:0',
});

Using @anthropic-ai/bedrock-sdk 0.10.4 and @anthropic-ai/sdk 0.29.2, but the error seems to come from Bedrock rather than the SDK.

@dandv dandv changed the title Bedrock: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row error after the silent change to combining consecutive turns Bedrock: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row error after the silent change to combine consecutive turns Oct 19, 2024
@rabsef-bicrym
Copy link

I get this error with sdk 0.32.1, not with bedrock.

  51 |   for (let i = 1; i < result.length; i++) {
  52 |     if (result[i].role === result[i-1].role) {
> 53 |       throw new Error('Invalid message sequence - messages must alternate between user and assistant');
     |            ^
  54 |     }
  55 |   }
  56 |

@dandv
Copy link
Contributor Author

dandv commented Nov 18, 2024

Can still reproduce with
"@anthropic-ai/bedrock-sdk": "^0.11.2",
"@anthropic-ai/sdk": "^0.32.1",

 error: {
    message: 'messages: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row'
  }

@dandv
Copy link
Contributor Author

dandv commented Dec 6, 2024

To clarify, I'm seeing different behaviors from Anthropic and Bedrock on the same prompt with identical consecutive roles.

import Anthropic from '@anthropic-ai/sdk';
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk';

const clientAnthropic = new Anthropic();
const clientBedrock = new AnthropicBedrock();

const haiku = {
  anthropic: 'claude-3-haiku-20240307',
  bedrock: 'anthropic.claude-3-haiku-20240307-v1:0',
};

async function consecutiveRoles(client: Anthropic | AnthropicBedrock, model = haiku.anthropic) {
  const response = await client.messages.create({
    max_tokens: 1,
    stream: false,
    messages: [
      { role: 'user', content: 'What is the color of' },
      { role: 'user', content: 'the sky? Answer with only one word.' },
    ],
    model,
  });
  console.log('Response content: ', response.content);
}

await consecutiveRoles(clientAnthropic, haiku.anthropic);  // [ { type: 'text', text: 'Blue' } ]
await consecutiveRoles(clientBedrock, haiku.bedrock);  // Error: 400 messages: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row

Per the Messages API docs that "Consecutive user or assistant turns in your request will be combined into a single turn.", it seems that Bedrock might not be running the same version of the API handlers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants