Skip to Main Content
Feature Request FR-3929
Product Area AI Features
Status CLOSED

2 Voters

Parse OpenAI Chat completions in REST Data Source

ino.laurensse Public
· Jul 25 2024

Idea Summary
Parse OpenAI chat response when creating a REST Data Source with v1/chat/completions endpoint.

Use Case
I noticed this when I was trying to upgrade to a new model (APEX 24.1). The newer models are all(?) chat models (they return an error when trying to use v1/completions instead of v1/chat/completions).

OpenAI responses are parsed correctly when creating a REST Data Source with the legacy v1/completions endpoint. E.g. using this old model

{"prompt": "#PROMPT#", "model": "gpt-3.5-turbo-instruct", "temperature": 0.7, "max_tokens": 200}

which returns

{
 "id" : "cmpl-.....",
 "object" : "text_completion",
 "created" : 1721931469,
 "model" : "gpt-3.5-turbo-instruct",
 "choices" :
 [
   {
     "text" : "\n\nOracle APEX (Application Express) is a web-based development platform for building data-driven applications. etc. ",
     "index" : 0,
     "logprobs" : null,
     "finish_reason" : "stop"
   }
 ],
 "usage" :
 {
   "prompt_tokens" : 4,
   "completion_tokens" : 118,
   "total_tokens" : 122
 }
}

The REST Data Source automatically retrieves “text” for the Parsed Data.

The new v1/chat/completions return the answer in “content”:

{
 "id" : "chatcmpl-....",
 "object" : "chat.completion",
 "created" : 1721931275,
 "model" : "gpt-4o-mini-2024-07-18",
 "choices" :
 [
   {
     "index" : 0,
     "message" :
     {
       "role" : "assistant",
       "content" : "Oracle APEX (Application Express) is a low-code development platform provided by Oracle Corporation that enables developers to build scalable, secure enterprise applications with minimal coding effort. …."
     },
     "logprobs" : null,
     "finish_reason" : "stop"
   }
 ],
 "usage" :
 {
   "prompt_tokens" : 21,
   "completion_tokens" : 425,
   "total_tokens" : 446
 },
 "system_fingerprint" : "fp_...."
}

Preferred Solution (Optional)
Retrieve either text or content as the Parsed Data.

This is a great idea! You can already achieve this in APEX today with a slightly different approach.

Comments

Comments

  • vincent morneau Admin OP 10 months ago

    This is doable today. In APEX 24.1 it's possible to consume using nested structure with JSON Data Sources, which can create a data profile from it. If you need help with the implementation, post a link in the forums and I'll make sure someone replies to it.

  • ino.laurensse OP 10 months ago

    @vincent morneau I am already using the chat completions, no problem. 

    Even the deprecated completions endpoint is nested, but it looks like the APEX team did something special for this response. The Parsed Data is automatically set to the value of “text”. If the APEX team indeed did something special for this response, they could do it for chat completions. That was the gist of the idea.