RAG with the IONOS AI Model Hub: Document search completely from Germany

The IONOS AI Model Hub can be used to set up an AI document search based on the RAG principle, whose language and embedding models run in German data centers. You operate the vector database, such as Qdrant, and the application on your own servers in Germany. This means that neither documents nor questions leave the German legal area.

Many companies want to make their knowledge searchable via AI, but shy away from US services. This article shows an architecture that remains entirely in Germany. RAG explains the basics of RAG in simple terms.

The building blocks

This is how a request works

1. An employee asks a question in the chat interface.
2. The question is converted to a vector via the AI Model Hub.
3. Qdrant searches for suitable sections – filtered by the employee's rights.
4. Optionally, a reranker re-evaluates the hits.
5. The best sections go with the question to the language model in the AI Model Hub.
6. The answer comes back with the source.

Which models are suitable

• Embeddings: For German texts, a multilingual model is required, such as bge-m3 (€0.02 per million tokens) or paraphrase-multilingual-mpnet-base-v2 (€0.01).
• Reranker: Qwen3-VL-Reranker-8B (€0.04 per million tokens) improves the hit order.
• Language model: Mistral Small 24B as a low-cost all-rounder, gpt-oss-120b or Llama 3.3 70B for more demanding answers.

Net prices according to the IONOS price list, as of October 2026. More on this in the article IONOS AI Model Hub Prices.

Why the vector database itself is operated

According to the price list, IONOS lists the former "Document Collections" as an old offering that is no longer available for new deployments. For new projects, we therefore operate the vector database ourselves – with Qdrant on a server in Germany. This has advantages: full control over filters and authorizations, hybrid search, and no risk of switching if the offer changes again.

Data protection

IONOS states that it operates the AI Model Hub in Germany in compliance with the GDPR and does not use customer data for training. To use it, you need a data processing agreement with IONOS; your own measures apply to the parts you operate yourself. The article AI document search and GDPR describes what is important.

Effort realistically estimated

The model costs are low; the effort is involved in connecting the document sources, the sensible decomposition, authorizations and tests. This is exactly what our AI document search is: built with the IONOS AI Model Hub, Qdrant and n8n, operated in Germany.

Counselling and accompanied implementation is eligible for funding: Through INQA coaching, the federal government covers 80% of the consulting costs of up to €1,200 per day for up to 12 days, i.e. up to €11,520 subsidy. In Mecklenburg-Western Pomerania and the other new federal states, BAFA consulting funding is also eligible at 80% until 31.12.2026 (50% in the old federal states). Funding is provided for consulting, not software. The funding check shows which program fits.

Frequently Asked Questions

Can I operate a RAG system entirely in Germany?

Yes. Voice and embedding models run in the IONOS AI Model Hub in German data centers, vector database, and application on servers in Germany.

Does IONOS offer its own vector database?

According to the price list, the former document collections are no longer available for new deployments. For new projects, a self-operated vector database such as Qdrant is recommended.

Which embedding model is suitable for German texts?

A multilingual model such as bge-m3 or paraphrase-multilingual-mpnet-base-v2. Purely English models deliver worse hits for German documents.

How much does a RAG system with IONOS cost to operate?

The model costs are often in the cent to low euro range per month for typical use. In addition, there are servers for vector database and application as well as maintenance.

Would you like to make your company knowledge searchable with AI from Germany? Talk to us via the contact page.