I am aware LLM's in general are crap at doing even simple mathematics, but I did hope a copilot could answer questions that requires simple maths. My test is a total failure, and I would appreciate any ideas as to how to make a copilot answer questions that requires only very simple maths.
For my test i added sales transaction data as Copilot knowledge: I uploaded a CSV with "faked" sales transactions, with columns for purchase date, product sold, product price and in what city the product was sold.
I then first asked the copilot to display the total sales value per city in March (where it listed sales value for Gothenburg as 505). I then asked for the total sales value in March for Gothenburg, and got 4989. Answering the same question in two different ways gave a 10X difference in the answer. Clearly not useful for anything.
See screenshot below for reference.
So, I would appreciate any ideas as to how to make a copilot being able to do at least simple value-filtering (e.g. by date-span) and maths (e.g summarizing the total or presenting the average value).
I am in control of the back-end with the SQL Server database, so we could develop a REST API that the copilot can call to get answers to specific questions. I do however want the copilot to be able to answer various types of questions sales managers may have, that is unknown in advance. E.g. I want the copilot to support answering many types of questions that may not be known in advance at development time, so making a REST API in the backend doesnt really work.
Any ideas most appreciated!
