Dev Update: Bing vs. Google… The Shocking Betrayal
We’re back with a new boodleGPT development update. Once again, you’ll hear from Kisa Brostrom, Chief Data Officer at boodleAI, and Ansel Teng, Chief Technology Officer at boodleAI, this week as they share boodleGPT’s progress, and offer some insights into the fun, yet sometimes frustrating, world of LLM development.
Data Team Update
Chief Data Officer of boodleAI, Kisa is affectionately referred to as “Queen Kisa”, responsible for all things data. A mechanical engineer by education, she’s built her career on growing data teams from scratch and developing useful, practical solutions. When she’s not herding cattle in her minivan, she’s raising the next generation of data artists (9, 6, 3) on a farm in Oklahoma. Connect with Kisa on LinkedIn.
Anyone that’s spent anytime with children will often tell you: “Kids are brutally honest”. I’m not sure I agree with that. A lot of times kids are just brutal, honesty isn’t a requirement 😂. Turns out computers and LLMs are the same.
After spending a week or so improving the quality of responses it gives, we’re now focusing on how to make boodleGPT more flexible by adding in new capabilities, like web search and first party data, and allowing for any combination of LLMs, datasources, and personality.
Switching LLM’s in no easy task; it’s a huge undertaking. Getting one LLM to answer correctly is hard, generalizing so that ANY LLM can work is the Mount Everest of MLOps and Data Engineering. Each LLM wants to be initialized in their specific way; some work with zero shot, others need chain of thought reasoning. It’s definitely a bit like chaperoning a preschool field trip to the zoo; sounds like fun for everyone else, except the chaperone.
The integration of first party data is actually going incredibly smoothly. It’s just a lot of infrastructure: ingestion, mapping, reformatting, aggregation, transferring. Luckily, we’ve spent the past 5 years building that infrastructure, so it’s provided us a good head start and foundation to build off of.
And finally, web search. For something that we all do several times a day, it’s actually EXCEPTIONALLY nuanced (aka difficult) to build into an LLM. You have to decide what to search, how many links to search, do you use the snippets or scrape the pages, and how do you decide whether or not the LLM has enough data to answer the question.
As a part of the development process, we’ve actually been testing it both with Bing API and Google API and I have to say I’m shocked. I’ve never used the Bing search engine in my everyday life; everything I do is on Google. However, in a shocking betrayal, Google refuses to acknowledge my existence. Bing, however, answers perfectly every time with a lot of nuance as well.
I’m not saying I’m mad, but I am disappointed in you, Google.
Tech Team Update
Chief Technology Officer of boodleAI, Ansel is a senior technology leader since the dot com days. He has a Ph.D. in computer science and has been managing software product and system development in organizations of all sizes, from small startup to global enterprise. Connect with Ansel on LinkedIn. He loves to connect with fellow technology leaders.
A new version of the boodleGPT Alpha was released earlier last week. It incorporates a new engine that makes better use of the boodle predictive insights and a number of user experience enhancements:
- Revised page layout to better utilize the full height of the screen for the chat responses. It also provides a much better mobile experience and supports full screen capture and printing of the entire session;
- Enhancements to the user feedback widget. A copy to clipboard feature was added to allow users to take the response to another application easily. It also asks for comments upon rating to encourage detailed user feedback.
- Invitation and account maintenance are now boodleGPT branded to improve the consistency throughout the experience.
With these quick wins behind us, we now focus on the next set of big capabilities, such as the ability to bring in organization data so that boodleGPT can answer questions specific to their donors and customers. Ideas are also brewing on how best to provide users the ability to use boodleGPT to automate their routine tasks. Stay tuned. It will be exciting!
