Unanswered

Custom AI Prompt throwing error "The prompt contained more input tokens than the model supports"

(1) Share

Report

Posted on by Usman2

124

Hi

I am creating a custom prompt to get insights from meeting agenda. I am using custom Dataverse table as the data source of the prompt. My table has 584 records, and the agenda title data type is single line. I am getting "The prompt contained more input tokens than the model supports" error.

I want to understand is that why is this limit exceeding and how can I make sure that I am staying within the limit?

Categories:

AI Builder

I have the same question (0)

All responses (5)

Answers (1)

Verified answer

CU05111601-0 1,091 Super User 2024 Season 2 on at

Like (2)

Report

Hi @Usman2 , it is exceeding the limit because it tries to put ALL 584 records into your prompt. That boils down to a LOT of text which makes it exceed the limit. I would recommend to limit the records you are providing to the system.

You might want to add a input variable and feed this input variable via a Power Automate flow. In your power automate flow make sure to filter only the records for the previous months. This will greatly reduce the amount of text you are giving to the prompt which prevents the AI from reaching the limit.

Was this reply helpful? Yes No
Usman2 124 on at

Like (1)

Report

Hi @Tjan

Thank you for the reply. Isn't there any method to somehow limit the columns from the database which will reduce the number of input tokens. Like in PowerApps we have the ShowColumns function to limit the columns we want to collect.

Was this reply helpful? Yes No
CU05111601-0 1,091 Super User 2024 Season 2 on at

Like (2)

Report

HI @Usman2 ! Yes there is! But the prompt interface is not the place to do this. In Power Automate you have almost full freedom to do so!

Was this reply helpful? Yes No
Usman2 124 on at

Like (1)

Report

@Tjan Thanks for sharing this information.

Was this reply helpful? Yes No
AlexAguirreF on at

Like (2)

Report

Hello!
I still have doubts about it.
@Tjan when you says: "That boils down to a LOT of text which makes it exceed the limit", What do you mean?? Whats is the token limit for a input prompt??
On the documentation i can find:
- GPT 3.5 support 16K tokens, and GPT 4 support 32Ktokens (https://learn.microsoft.com/en-us/ai-builder/prompt-modelsettings)
Apart fromthat, I used OpenAI's "Tokenizer web" (https://platform.openai.com/tokenizer), and I calculated the total tokens for my prompt, including the Dataverse table, and it counts 12k tokens approximately.

Where is the trick??

Was this reply helpful? Yes No