Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenAI: unexpected token at '```json #590

Open
yshmarov opened this issue Jul 15, 2024 · 5 comments
Open

OpenAI: unexpected token at '```json #590

yshmarov opened this issue Jul 15, 2024 · 5 comments

Comments

@yshmarov
Copy link

I get an error when trying to translate with OpenAI

Users/yshmarov/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/json-2.7.2/lib/json/common.rb:220:in `parse': unexpected token at '```json (JSON::ParserError)
[
  "Wir werden Sie kontaktieren, um Ihnen mitzuteilen, dass Ihre E-Mail-Adresse auf X__0 geändert wurde.",
  "Erwartetes Ergebnis",
  "Eo",
  "ko",
  "Sie können den API-Schlüssel finden",
  "hier",
  "Bezahlt",
  "Haben Sie bereits ein SR-Konto?",
  "SR arbeitet mit Partnern zusammen, um Ihre Bank sicher zu verbinden. Dies sind regulierte Organisationen, die sicherstellen, dass Ihre Daten geschützt sind.\n\nDies entspricht d

OPENAI_MODEL=gpt-4o OPENAI_API_KEY=sk-***foobar i18n-tasks translate-missing --backend=openai

Does anybody else have this issue?
@yshmarov
Copy link
Author

the above issue seems to be related to gpt-4o.

However when I run gpt-3.5-turbo, I get another error:

 i18n-tasks translate-missing --backend=openai
warning: parser/current is loading parser/ruby33, which recognizes 3.3.4-compliant syntax, but you are running 3.3.0.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
#StandWithUkraine
/Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:98:in `next': iteration reached an end (StopIteration)
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:98:in `parse_value'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:90:in `block in parse_value'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:90:in `map'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:90:in `parse_value'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:63:in `from_values'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:46:in `fetch_translations'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:35:in `block in translate_pairs'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:34:in `each'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:34:in `map'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:34:in `translate_pairs'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:17:in `block in translate_forest'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/data/tree/nodes.rb:16:in `each'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/data/tree/nodes.rb:16:in `each'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:16:in `inject'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translators/base_translator.rb:16:in `translate_forest'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/translation.rb:21:in `translate_forest'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/command/commands/missing.rb:51:in `translate_missing'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/command/commander.rb:26:in `run'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:51:in `block in run'
	from /Users/yaroslavshmarov/.gem/ruby/3.3.0/gems/i18n-1.14.5/lib/i18n.rb:351:in `with_locale'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:49:in `run'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:18:in `block in start'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:211:in `auto_output_coloring'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:17:in `start'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/lib/i18n/tasks/cli.rb:11:in `start'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/i18n-tasks-1.0.14/bin/i18n-tasks:15:in `<top (required)>'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/bin/i18n-tasks:25:in `load'
	from /Users/yaroslavshmarov/.rbenv/versions/3.3.0/bin/i18n-tasks:25:in `<main>'

@davidwessman
Copy link
Collaborator

Could you add e.g. your i18n-tasks.yml and some more info that would help recreate it?

@yshmarov
Copy link
Author

Could you add e.g. your i18n-tasks.yml and some more info that would help recreate it?

Hi David! I am using exclusively the defaults in the yml file, only uncommenting open-ai:

base_locale: en
data:
  read:
  write:
  external:
  yaml:
    write:
      line_width: -1
search:
  exclude:
    - app/assets/images
    - app/assets/fonts
    - app/assets/videos
    - app/assets/builds
translation:
  openai_api_key: "sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  openai_model: "gpt-3.5-turbo"

It used to work before

@yshmarov
Copy link
Author

possibly it does not parse my YML file correctly? I have 6000 lines in en.yml....

@davidwessman
Copy link
Collaborator

possibly it does not parse my YML file correctly? I have 6000 lines in en.yml....

Yeah, probably some markup that is not working :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@davidwessman @yshmarov