ingest

RAG document ingestion pipeline

Go RAG & Memory GitHub

Process markdown into semantic vector embeddings stored in PostgreSQL+pgvector. Add, list, delete, and search documents for retrieval-augmented generation.

Features

  • Ingest markdown files into vector embeddings
  • PostgreSQL + pgvector storage
  • Ollama embedding via nomic-embed-text
  • Add, list, delete, and search operations
  • Language filtering for search results

Install

go install github.com/hegner123/ingest@latest

The Problem: AI agents can't search your documentation by meaning

# You have 200 markdown docs across your project.
# Grep finds exact strings, not concepts.
# "How do I handle authentication?" returns nothing
# because no file contains that exact phrase.

Solution

$ ingest add --path ./docs --language go --recursive

Output

{"ingested":15,"skipped":2,"errors":0,"language":"go","path":"./docs"}

Comparison

MetricValue
Search typeSemantic (meaning-based, not keyword)
Embedding modelnomic-embed-text via Ollama (768 dims)
StoragePostgreSQL + pgvector