Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blog post for OpenTelemetry Generative AI updates #5575

Open
wants to merge 93 commits into
base: main
Choose a base branch
from

Conversation

drewby
Copy link
Member

@drewby drewby commented Nov 9, 2024

Title: OpenTelemetry for Generative AI

This blog post introduces enhancements to OpenTelemetry specifically tailored for generative AI technologies, focusing on the development of Semantic Conventions and the Python Instrumentation Library.

Samples are in Python

SIG: GenAI Observability

Sponsors: @tedsuo @lmolkova

Closes: #5581

@drewby drewby requested a review from a team as a code owner November 9, 2024 00:40
@opentelemetrybot opentelemetrybot requested a review from a team November 9, 2024 00:40
@github-actions github-actions bot added the blog label Nov 9, 2024
Co-authored-by: Liudmila Molkova <[email protected]>
drewby and others added 2 commits November 9, 2024 10:52
Co-authored-by: Liudmila Molkova <[email protected]>
Co-authored-by: Liudmila Molkova <[email protected]>
Copy link
Member

@theletterf theletterf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool one! Added some comments.

content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
@opentelemetrybot opentelemetrybot requested a review from a team November 10, 2024 23:01
@codefromthecrypt
Copy link

The below is about if we can make it zero code or not, and what's remaining to do so:


I want to run dotenv run -- opentelemetry-instrument python main.py and not have any telemetry code in the foreground or even noticed (zero code approach)

I have a similar example, I've tried locally, and I don't see a way to implicitly configure the logging provider, yet. I'm not sure if we want to make a hybrid to reduce the amount of code or just leave the explicit tracing and logging stuff in until logging can be env configured.

cc @anuraaga and @xrmx in case I got below wrong also

requirements

openai~=1.54.3

opentelemetry-sdk==1.28.1
opentelemetry-exporter-otlp-proto-http==1.28.1
opentelemetry-distro==0.49b1
opentelemetry-instrumentation-openai-v2==2.0b0

env

# Override default ENV variables for Ollama
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=unused
CHAT_MODEL=qwen2.5:0.5b

# Set to false or remove to disable log events
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_SERVICE_NAME=python-opentelemetry-openai
OTEL_METRIC_EXPORT_INTERVAL=100
OTEL_BSP_SCHEDULE_DELAY=100

Best I could manage was to add hooks only for the log/event stuff

import os

from openai import OpenAI

# NOTE: OpenTelemetry Python Logs and Events APIs are in beta
from opentelemetry import _logs, _events
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._events import EventLoggerProvider

from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter

_logs.set_logger_provider(LoggerProvider())
_logs.get_logger_provider().add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))
_events.set_event_logger_provider(EventLoggerProvider())

def main():
    client = OpenAI()
    messages = [
        {
            "role": "user",
            "content": "Answer in up to 3 words: Which ocean contains the falkland islands?",
        },
    ]
    model = os.getenv("CHAT_MODEL", "gpt-4o-mini")
    chat_completion = client.chat.completions.create(model=model, messages=messages)
    print(chat_completion.choices[0].message.content)


if __name__ == "__main__":
    main()

then, I get a warning about overriding the event provider, but at least the events do show up

$ dotenv run -- opentelemetry-instrument python main.py
Overriding of current EventLoggerProvider is not allowed
Indian Ocean

Copy link

@codefromthecrypt codefromthecrypt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess what's out of scope is metrics, as they aren't implemented yet. One concern is that folks follow this, then later when metrics are implemented, they need more instructions, or people need to remember to go back and change the docs etc.

Since there are recent developments like asyncapi etc going in fairly quickly, if metrics were added quickly also, would it make sense to hold the blog until they are released? or would it make more sense to do a second blog and revisit the setup instructions once that's supported?

@drewby
Copy link
Member Author

drewby commented Nov 11, 2024

I guess what's out of scope is metrics, as they aren't implemented yet. One concern is that folks follow this, then later when metrics are implemented, they need more instructions, or people need to remember to go back and change the docs etc.

Since there are recent developments like asyncapi etc going in fairly quickly, if metrics were added quickly also, would it make sense to hold the blog until they are released? or would it make more sense to do a second blog and revisit the setup instructions once that's supported?

Metrics are discussed as part of the semantic conventions, but correct, they are not yet implemented in the library yet. I think it's worth getting an article out there earlier than later. We might even attract some contributors for the Metrics implementation.

@codefromthecrypt
Copy link

We might even attract some contributors for the Metrics implementation.

good point!

Copy link
Member

@svrnm svrnm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great read, thanks for writing this!

content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
---
title: OpenTelemetry for Generative AI
linkTitle: OpenTelemetry for Generative AI
date: 2024-11-09
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting this comment here to keep an eye on setting the date right when we finally publish. Do not resolve.

Suggested change
date: 2024-11-09
date: 2024-11-09

@svrnm svrnm requested a review from tedsuo November 11, 2024 09:00
@opentelemetrybot opentelemetrybot requested a review from a team November 11, 2024 09:03
Copy link
Contributor

@chalin chalin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial drive-by comments, with a suggestion for fixing the build:

> hugo --cleanDestinationDir -e dev --buildDrafts --baseURL https://deploy-preview-5575--opentelemetry.netlify.app/ --minify
...
Error: error building site: process: readAndProcessContent: "/opt/build/repo/content/en/blog/2024/otel-generative-ai/index.md:8:1": failed to unmarshal YAML: yaml: line 8: did not find expected ',' or ']'
Failed during stage 'building site': Build script returned non-zero exit code: 2 (https://ntl.fyi/exit-code-2)
...

content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
content/en/blog/2024/otel-generative-ai/index.md Outdated Show resolved Hide resolved
@opentelemetrybot opentelemetrybot requested a review from a team November 22, 2024 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Blog post for OpenTelemetry Generative AI updates