Prepare Ollama and LanceDB

This is part of the Use RAG with Continue.dev series.

Prerequisites

I kept in line with the general prerequisites for the previous steps (python 3, uv for package management). In addition, I will assume you have ollama installed. Check here for different installation methods.

Setup

Ollama

Once Ollama is installed, you need to download the models which will be used:

1. deepseek-coder-v2:latest - for the actual prompting
2. mxbai-embed-large:latest - to generate and query embeddings

Note: I have not included a reranking model, mainly because the results I got were OK, and because ollama itself does not support reranking (at the moment of writing this).

You do this via {sh} ollama pull model_name.

Now have have the models needed, and we need to install the ollama API:

uv add ollama

LanceDB

LanceDB is a sqlite-like vector database, in the sense that you don't need a dedicated server. I'm using this because is suggested in the continue.dev documentation as easy-to-use. The command I used is:

uv add lancedb
uv add scikit-learn

Requests

If you haven't already installed it from the previous chapters, add the requests package with:

uv add requests

Now we have all dependencies and can start with indexing the code, one file at a time.

Bonus: Qdrant

After playing for a while with LanceDB, I've decided to try Qdrant. The main difference from my perspective is that LanceDB is an embedded database (like sqlite), while Qdrant is a "proper" one, where you have a server 😄. For my purposes, I've used a docker image that I installed locally on my pc:

docker run --name=qdrant \
   --hostname=9d90938f89b0 \
   --user=0:0 \
   --volume /home/laur/dev/ai/rag/qdrant_data:/qdrant/storage \
   --network=n8n_default \
   --workdir=/qdrant \
   -p 6333:6333 \
   -p 6334:6334 \
   --expose=6335 \
   --restart=always \
   --runtime=runc \
   qdrant/qdrant:latest \
   ./entrypoint.sh

Now, all I have to do is to connect to it from python. In my package, I've added:

uv add qdrant-client

HTH,