 |
 |
Gary Price did a presentation for CDL last week, the bullet points of which are here: bit.ly/CDLgpt (warning, that's a bit of a firehose), Gary mentioned RAG a bit. Based on his suggestion, I've been trying out phind.com which uses RAG. Phind works as both an AI chat, as well as a search engine. I kicked the tires a bit and asked it a question that I know had potential for hallucination, and the AI did not fall for my bait... it correctly reported that the functionality I was looking for did not exist. Then it offered to write a script to add the missing functionality, and as AIs are wont to do, it used a Python library that doesn't exist. :-) But I managed to steer the conversation in a direction where it could write a working script. In checking with Gary to see if it was ok to share the link above, he asked to to mention that he will be updating the page and "I'm always sharing a lot of material on infoDOCKET.com and ARL Day in Review." [1] --Hardy 1. https://www.arl.org/category/day-in-review/ On Mon, Feb 26, 2024 at 3:16 PM Eric Lease Morgan < [log in to unmask]> wrote: > Who out here in Code4Lib Land is practicing with either one or both of the > following things: 1) fine-tuning large-language models, or 2) > retrieval-augmented generation (RAG). If there is somebody out there, then > I'd love to chat. > > When it comes to generative AI -- things like ChatGPT -- one of the first > things us librarians say is, "I don't know how I can trust those results > because I don't know from whence the content originated." Thus, if we were > create our own model, then we can trust the results. Right? Well, almost. > The things of ChatGPT are "large language models" and the creation of such > things are very expensive. They require more content than we have, more > computing horsepower than we are willing to buy, and more computing > expertise than we are willing to hire. On the other hand there is a process > called "fine-tuning", where one's own content is used to supplement an > existing large-language model, and in the end the model knows about one's > own content. I plan to experiment with this process; I plan to fine-tune an > existing large-language model and experiment with it use. > > Another approach to generative AI is called RAG -- retrieval-augmented > generation. In this scenerio, one's content is first indexed using any > number of different techniques. Next, given a query, the index is searched > for matching documents. Third, the matching documents are given as input to > the large-language model, and the model uses the documents to structure the > result -- a simple sentence, a paragraph, a few paragraphs, an outline, or > some sort of structured data (CSV, JSON, etc.). In any case, only the > content given to the model is used for analysis, and the model's primary > purpose is to structure the result. Compared to fine-tuning, RAG is > computationally dirt cheap. Like fine-tuning, I plan to experiment with RAG. > > To the best of my recollection, I have not seen very much discussion on > this list about the technological aspects of fine-tuning nor RAG. If you > are working these technologies, then I'd love to hear from you. Let's share > war stories. > > -- > Eric Morgan <[log in to unmask]> > Navari Family Center for Digital Scholarship > University of Notre Dame >