Skip to content

Commit

Permalink
New image analysis example
Browse files Browse the repository at this point in the history
  • Loading branch information
ddebowczyk committed Oct 7, 2024
1 parent 9205bc3 commit e287a51
Show file tree
Hide file tree
Showing 3 changed files with 82 additions and 0 deletions.
Binary file added docs/images/car-damage.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/A05_Extras/ImageCarDamage/car-damage.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
82 changes: 82 additions & 0 deletions examples/A05_Extras/ImageCarDamage/run.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
title: 'Image processing - car damage detection'
docname: 'image_car_damage'
---

## Overview

This is an example of how to extract structured data from an image using
Instructor. The image is loaded from a file and converted to base64 format
before sending it to OpenAI API.

In this example we will be extracting structured data from an image of a car
with visible damage. The response model will contain information about the
location of the damage and the type of damage.

## Scanned image

Here's the image we're going to extract data from.

![Receipt](/images/car-damage.jpg)


## Example

```php
<?php
$loader = require 'vendor/autoload.php';
$loader->add('Cognesy\\Instructor\\', __DIR__ . '../../src/');

use Cognesy\Instructor\Extras\Image\Image;
use Cognesy\Instructor\Instructor;
use Cognesy\Instructor\Schema\Attributes\Description;

enum DamageSeverity : string {
case Minor = 'minor';
case Moderate = 'moderate';
case Severe = 'severe';
case Total = 'total';
}

enum DamageLocation : string {
case Front = 'front';
case Rear = 'rear';
case Left = 'left';
case Right = 'right';
case Top = 'top';
case Bottom = 'bottom';
}

class Damage {
#[Description('Identify damaged element')]
public string $element;
/** @var DamageLocation[] */
public array $locations;
public DamageSeverity $severity;
public string $description;
}

class DamageAssessment {
public string $make;
public string $model;
public string $bodyColor;
/** @var Damage[] */
public array $damages = [];
public string $summary;
}

$assessment = (new Instructor)->respond(
input: Image::fromFile(__DIR__ . '/car-damage.jpg'),
responseModel: DamageAssessment::class,
prompt: 'Identify and assess each car damage location and severity separately.',
model: 'gpt-4o',
options: ['max_tokens' => 4096]
);

dump($assessment);
assert($assessment->make === 'Toyota');
assert($assessment->model === 'Prius');
assert($assessment->bodyColor === 'white');
assert(count($assessment->damages) > 0);
?>
```

0 comments on commit e287a51

Please sign in to comment.