Nvidia’s free tool lets you create your own chatbot right on your PC

Your homebrew chatbot can pull answers using your own files — without putting your data at risk.
Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week

Nvidia has released a free tool you can use to create a custom chatbot that quickly searches your computer for answers to your questions — all while ensuring your private data stays private.

The challenge: Large language models (LLMs) learn to understand and generate text written in natural language by “reading” tons of data. The LLM behind OpenAI’s ChatGPT, for example, was trained on text pulled from the internet, as well as other sources.

Because an LLM’s “knowledge” is limited to the content included in its training data, the original ChatGPT couldn’t speak with authority on anything that happened after the cutoff date for its training data (January 2022).

A custom chatbot: Nvidia — the third biggest tech company in the world — has now released a free demo of a tool, called Chat with RTX, that lets you easily customize an open-source LLM, such as Meta’s Llama, with text files and videos.

You can give your custom chatbot access to a folder of PDFs on your computer, for example, and then ask it questions related to their content. If you feed it a link to a YouTube playlist, it can hunt through the videos’ transcripts for answers to your questions about the clips.

Nvidia's Chat With RTX tool open on a computer screen
Nvidia

While you could replicate this to an extent with ChatGPT — by copying and pasting text from a personal file into a chat before asking questions about it, for example — that AI does all of its processing in the cloud, meaning you’d be risking someone gaining access to the information. 

Besides that, cloud-based AIs usually have hard limits on how much data you can prompt them with at any given time, so even one long PDF file might be too long for it to read.

Chat with RTX is different. It’s free, so you don’t need a subscription, and it runs directly on your Windows PC. That not only protects your privacy, but can potentially lead to faster answers, as you aren’t beholden to busy servers.

The cold water: Chat with RTX can’t run on just any PC. Your system will need to meet Nvidia’s hardware requirements: “In addition to a GeForce RTX 30 Series GPU or higher with a minimum 8GB of VRAM, Chat with RTX requires Windows 10 or 11, and the latest NVIDIA GPU drivers.”

Early reviews suggest the tool is still a bit buggy, too. 

When one reviewer fed their custom chatbot a video link, it downloaded a transcript for a different video, and in another review, it answered a question correctly, but cited the wrong source for its answer. Imperfections are to be expected with a free demo, though.

The big picture: Nvidia is already a key player in the AI revolution. As of February 2023, it made 95% of the graphics cards needed to train and deploy chatbots, and more recently, it’s been developing and releasing hardware purpose-built for running generative AIs locally.

While it has released generative AI software, it’s been geared toward enterprise customers. If Nvidia keeps developing Chat with RTX in future versions, it could be hugely appealing to individuals looking for a safer, faster, cheaper AI.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week
Related
Cheap AI is causing a power shift in the world’s militaries
The militaries that leverage the low-cost capabilities of AI the most will have a decisive advantage in the future.
Media has a blind spot when covering the AI panic
When news outlets quote warnings of an impending AI catastrophe, they rarely mention the two main movements behind this narrative.
How AI could usher in The New Enlightenment
AI could trigger a civilization-scale change for humanity the same way the steam engine helped usher in The Enlightenment 250 years ago.
The missing tech case for how we create an era of abundance
AI and other new technologies could make things that are costly and scarce today, cheap and abundant for all tomorrow.
Why America reinvents itself every 80 years — and is doing so again
Three separate theories help explain why America enters a period of great progress every 80 years — and why another is coming soon.
Up Next
A man working on an old typewriter in a workshop, showcasing his trust in traditional methods of communication.
Subscribe to Freethink for more great stories