“The Assistants API allows you to build AI assistants within your own applications. An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries”, OpenAI.
Sounds great, so we are going to look at how we can use the new API to do data analysis on local data.
The Assistants API represents an approach that is an alternative to at least some uses of Retrieval Augmented Generation (RAG). So has RAG just been a stopgap measure, a temporary solution to the drawbacks of the current generation of LLMs? After all, LlamaIndex’s Jerry Liu has said that RAG is just a hack (albeit a powerful one).
Here are three specific problems inherent to LLMs that RAG currently addresses and that the Assistants API will also tackle:
- LLMs are out of date. It takes a lot of time and money (not to mention energy) to train a model, so the information that they were trained on could be a couple of years old.
- LLMs don’t know about your data. It is quite unlikely that your data were part of the training set for an LLM.
- LLMs hallucinate. Sometimes they will give entirely plausible responses that are totally false.
By providing the LLM with data that is relevant to your application you can reduce these problems.
For example, if you want the LLM to produce Streamlit code, you could give it data from the latest documentation to enable it to use new features of the framework. Or, if you want to do some analysis on some specific data, then clearly giving it that data is essential. And, finally, by providing relevant data to the LLM, you increase the chance of it providing a suitable response and thus reduce the possibility of it just making things up.
While RAG has been used to mitigate these issues, they are now also addressed by the new Assistants API. The RAG approach makes…