Setting up the Open WebUI knowledge base

a knowledge base in Open WebUI by uploading documents to a You can set up knowledge collection and assigning them to a model or individual chats. Open WebUI then searches for suitable text passages for each question and lets the language model answer them. How well this works depends less on the software than on clean documents, sensible division and clear rights.

Many teams first use Open WebUI as a ChatGPT replacement. The real added value arises when the chat can also answer questions about manuals, guidelines or product data. Basics of Open WebUI itself can be found in the article Open WebUI as a ChatGPT alternative.

How knowledge works in Open WebUI

Open WebUI works according to the principle of retrieval-augmented generation (RAG): Documents are broken down into sections and stored in a vector database. For a question, the most suitable sections are searched for and given to the model as a basis. According to the documentation, Open WebUI supports a hybrid search consisting of keyword and meaning search with subsequent reevaluation of the hits.

A more detailed explanation of the principle can be found in the article RAG simply explained.

Five steps to knowledge collection

Define a purpose: One collection per topic – such as "Personnel manual", "Product data sheets", "Quality management". Not everything in one pot.
Select documents: only current, approved versions. Outdated versions lead to incorrect answers.
Prepare documents: readable PDFs or Word files, meaningful headings, no scanned images without text recognition.
Upload and assign: Create a collection, add files, assign a collection to a model or wizard.
Testing: asking ten typical questions, comparing answers with the sources, making improvements.

Rights: Who is allowed to use which knowledge?

Open WebUI knows groups and permissions per resource. A knowledge collection "Salary Guidelines" should only be visible to the HR department. Create groups and assign access to collections and models via groups. More about this in the article Open WebUI in the team.

Typical mistakes

Collections that are too large: Hundreds of mixed documents dilute the search.
Scanned PDFs without text: If it's not text, it can't be found.
No maintenance: If a manual is updated, the collection must also be updated.
Model that is too weak: Small models sometimes summarize found areas inaccurately.

When it is better to have your own RAG system

The built-in knowledge function is sufficient for manageable, manually maintained collections. If thousands of documents from an archive such as Paperless-ngx or Nextcloud are to be kept up to date automatically, with permissions from the source system, a separate connection with a vector database such as Qdrant is more robust. This is exactly what our AI document search is.

We set up Open WebUI with knowledge collections, choose the right model and train your team.

Counselling and accompanied implementation is eligible for funding: Through INQA coaching, the federal government covers 80% of the consulting costs of up to €1,200 per day for up to 12 days, i.e. up to €11,520 subsidy. In Mecklenburg-Western Pomerania and the other new federal states, BAFA consulting funding is also eligible at 80% until 31.12.2026 (50% in the old federal states). Funding is provided for consulting, not software. The funding check shows which program fits.

Frequently Asked Questions

Can Open WebUI answer questions about your own documents?

Yes. Documents are uploaded to knowledge collections; Open WebUI searches for suitable sections and the model responds based on them.

What file formats does Open WebUI support?

Common formats such as PDF, Word and text files. Various extraction services can be integrated for text extraction. Scanned documents require text recognition beforehand.