On May 13th GPT-4o was released in preview in the Azure OpenAI Playground. No API access, only accessible in the browser. Today I was going into the playground to test something with the GPT-4o model, but could not find it. What had happened? Turnes out it suddenly was in GA and available to choose under Deploy model. The API version was 2024-05-13. I found some documentation on it here, but nothing about how to do the API requests.
I figured it might use the same request as the Vision models as I was going to do calls with text/image. I created a PowerShell script using the deployment I created with API version 2024-05-13, but I got the following error:
Looks like it could not find the endpoint. Why was that happening? My first thought was to see what API calls the playground was doing in the background, so I opened dev mode in my browser. Turns out the API version the playground was using was 2024-04-01-preview, even though the only model deployed was GPT-4o with version 2024-05-13. I tried switching to 2024-04-01-preview when doing the API calls in PowerShell, and it worked!
So I have created a PowerShell script that keeps the context of the conversation, analyze images and get responses based on them. I have written the script so you can both use local images and URL from online images. Below you have a couple examples of what is possible to do. All the way at the bottom I have the script I wrote.
Here i analyzed 5 images one by one, and then asked to list objects from all of them.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # This function takes a path to a file as a parameter and returns the base64 string of that file. | |
| function Get-Photo { | |
| param ( | |
| [Parameter(Mandatory=$true)] | |
| [string]$path | |
| ) | |
| [convert]::ToBase64String((get-content $path -Raw -AsByteStream)) | |
| } | |
| function Get-AzureOpenAIToken{ | |
| <# .SYNOPSIS | |
| Get an azure token for user or managed identity thats required to authenticate to Azure OpenAI with Rest API. | |
| Also construct the header if you are using an Azure OpenAI API key instead of Azure AD authentication. | |
| .PARAMETER ManagedIdentity | |
| Use this parameter if you want to use a managed identity to authenticate to Azure OpenAI. | |
| .PARAMETER User | |
| Use this parameter if you want to use a user to authenticate to Azure OpenAI. | |
| .PARAMETER APIKey | |
| Use this parameter if you want to use an API key to authenticate to Azure OpenAI. | |
| .EXAMPLE | |
| # Manually specify username and password to acquire an authentication token: | |
| Get-AzureOpenAIToken -APIKey "ghgkfhgfgfgkhgh" | |
| Get-AzureOpenAIToken -ManagedIdentity $true | |
| Get-AzureOpenAIToken -User $true | |
| .NOTES | |
| Author: Alexander Holmeset | |
| Twitter: @AlexHolmeset | |
| Website: https://www.alexholmeset.blog | |
| Created: 09-02-2023 | |
| Updated: | |
| Version history: | |
| 1.0.0 - (09-02-2023) Function created | |
| #> | |
| [CmdletBinding()] | |
| param ( | |
| [Parameter(Mandatory=$false)] | |
| [string]$APIKey, | |
| [Parameter(Mandatory=$false)] | |
| [string]$ManagedIdentity, | |
| [Parameter(Mandatory=$false)] | |
| [string]$User | |
| ) | |
| Process { | |
| $ErrorActionPreference = "Stop" | |
| if (Get-Module -ListAvailable -Name Az.Accounts) { | |
| # Write-Host "You have the Az.Accounts module installed" | |
| } | |
| else { | |
| Write-Host "You need to install the Az.Accounts module"; | |
| break | |
| } | |
| If(!$MyHeader){ | |
| If($ManagedIdentity -eq $true){ | |
| "managed" | |
| try { | |
| Connect-AzAccount -Identity | |
| $MyTokenRequest = Get-AzAccessToken -ResourceUrl "https://cognitiveservices.azure.com" | |
| $MyToken = $MyTokenRequest.token | |
| If(!$MyToken){ | |
| Write-Warning "Failed to get API access token!" | |
| Exit 1 | |
| } | |
| $Global:MyHeader = @{"Authorization" = "Bearer $MyToken" } | |
| } | |
| catch [System.Exception] { | |
| Write-Warning "Failed to get Access Token, Error message: $($_.Exception.Message)"; break | |
| } | |
| } | |
| If($User -eq $true){ | |
| "USER" | |
| try { | |
| Connect-AzAccount | |
| $MyTokenRequest = Get-AzAccessToken -ResourceUrl "https://cognitiveservices.azure.com" | |
| $MyToken = $MyTokenRequest.token | |
| If(!$MyToken){ | |
| Write-Warning "Failed to get API access token!" | |
| Exit 1 | |
| } | |
| $Global:MyHeader = @{"Authorization" = "Bearer $MyToken" } | |
| } | |
| catch [System.Exception] { | |
| Write-Warning "Failed to get Access Token, Error message: $($_.Exception.Message)"; break | |
| } | |
| } | |
| If($APIkey){ | |
| "APIKEY" | |
| $Global:MyHeader = @{"api-key" = $apikey } | |
| } | |
| } | |
| } | |
| } | |
| function Get-Chat { | |
| <# .SYNOPSIS | |
| Get a resposne from the chat endpoint in Azure OpenAI. | |
| .PARAMETER DeploymentName | |
| A deployment name should be provided. | |
| .PARAMETER ResourceName | |
| A Resource name should be provided. | |
| .PARAMETER Prompt | |
| A prompt name should be provided. | |
| .PARAMETER Token | |
| A token name should be provided. | |
| .EXAMPLE | |
| Get-Chat -DeploymentName $DeploymentName -ResourceName $ResourceName -maxtokens 1000 -prompt "What is the meaning of life?" -AssitantInstruction $AssitantInstruction | |
| .NOTES | |
| Author: Alexander Holmeset | |
| Twitter: @AlexHolmeset | |
| Website: https://www.alexholmeset.blog | |
| Created: 09-02-2023 | |
| Updated: | |
| Version history: | |
| 1.0.0 - (09-02-2023) Function created | |
| #>[CmdletBinding()] | |
| param ( | |
| [parameter(Mandatory = $false, HelpMessage = "Your azure openai deployment name")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$DeploymentName, | |
| [parameter(Mandatory = $false, HelpMessage = "your azure openai resource name")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$ResourceName, | |
| [parameter(Mandatory = $false, HelpMessage = "Your Azure OpenAI prompt")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$Prompt, | |
| [parameter(Mandatory = $false, HelpMessage = "Your Azure OpenAI service instructions")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$AssitantInstruction, | |
| [parameter(Mandatory = $true, HelpMessage = "Use photo?")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$ReadImage, | |
| [parameter(Mandatory = $true, HelpMessage = "Set number of max tokens")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$Maxtokens, | |
| [parameter(Mandatory = $true, HelpMessage = "Is image local or online?")] | |
| [ValidateNotNullOrEmpty()] | |
| [string]$ImageLocalOrOnline | |
| ) | |
| Process { | |
| $ErrorActionPreference = "Stop" | |
| $APIVersion = "2024-04-01-preview" | |
| # Construct URI | |
| $uri = "https://$ResourceName.openai.azure.com/openai/deployments/$DeploymentName/chat/completions?api-version=$ApiVersion" | |
| # Construct Body | |
| if ($ReadImage -eq $true) { | |
| If($ImageLocalOrOnline -like "online"){ | |
| $script:conversation += @" | |
| {"role": "user", "content": [{"type": "text", "text": "$Prompt"},{"type": "image_url","image_url": {"url": "$url"}}]}, | |
| "@ | |
| } | |
| else{ $script:conversation += @" | |
| {"role": "user", "content": [{"type": "text", "text": "$Prompt"},{"type": "image_url","image_url": {"url": "data:image/jpeg;base64,$photo"}}]}, | |
| "@ | |
| } | |
| } | |
| else { | |
| $script:conversation += @" | |
| {"role": "user", "content": [{"type": "text", "text": "$Prompt"}]}, | |
| "@ | |
| } | |
| $body = @" | |
| { | |
| "messages": [ | |
| $script:conversation | |
| {"role": "system", "content": "$assitantInstruction"} | |
| ], | |
| "max_tokens": $maxtokens, | |
| "stream": false | |
| } | |
| "@ | |
| try { | |
| $Global:Request = invoke-restmethod -Method POST -Uri $uri -ContentType "application/json" -Body $body -Headers $Global:MyHeader | |
| $script:conversation += @" | |
| {"role": "assistant", "content": "$($Request.choices.message.content)"}, | |
| "@ | |
| } | |
| catch [System.Exception] { | |
| $_ | |
| Write-Warning "Failed to to POST request: $($_.Exception.Message)"; break | |
| } | |
| "User: $Prompt" | |
| "Assistant: $($Request.choices.message.content)" | |
| } | |
| } | |
| Get-AzureOpenAIToken -APIKey "Enter API key here." | |
| $DeploymentName = "Enter deployment name here." | |
| $ResourceName = "Enter resource name here." | |
| $AssitantInstruction = "You are a helpfull assistant." | |
| #Path to the image to upload. | |
| $path = "Enter path to image here." | |
| $photo = Get-Photo -path $path | |
| #URL to image to upload. | |
| $url = "Enter URL to image here." | |
| $Prompt = "Please describe this photo." | |
| #If doing a request with an image, set ReadImage to true, and ImageLocalOrOnline to either local or online. | |
| Get-Chat -DeploymentName $DeploymentName -ResourceName $ResourceName -AssitantInstruction $AssitantInstruction -Prompt $Prompt -ReadImage $true -ImageLocalOrOnline local -maxtokens 4000 | |
A blog about automation and technologies in the cloud 