Quantcast
Channel: how can I count tokens before making api call? - Stack Overflow
Viewing all articles
Browse latest Browse all 3

how can I count tokens before making api call?

$
0
0
import { Configuration, OpenAIApi } from "openai"import { readFile } from './readFile.js'// Config OpenAI APIconst configuration = new Configuration({    organization: "xyx......",    apiKey: "abc.......",});// OpenAI API instanceexport const openai = new OpenAIApi(configuration);const generateAnswer = async (conversation, userMessage) => {    try {        const dataset = await readFile();        const dataFeed = { role: 'system', content: dataset };        const prompt = conversation ? [...conversation?.messages, dataFeed, userMessage] : [dataFeed, userMessage];        const completion = await openai.createChatCompletion({            model: "gpt-3.5-turbo",            messages: prompt        })        const aiMessage = completion.data.choices[0].message;        console.log(completion.data.usage)        return aiMessage    } catch (e) {        console.log(e)    }}export { generateAnswer };

I am trying to create chat bot, in which I provide datafeed in start which is business information and conversation history to chat apiI want to calculate tokens of conversation and reduce prompt if exceeds limit before making api callI have tried using gpt3 encoder to count tokens but i have array of objects not string in prompt


Viewing all articles
Browse latest Browse all 3

Latest Images

Trending Articles





Latest Images