We are super happy to launch the very first, early release of Swiftide. After working with Python based alternatives on Bosun.ai, the need arose to build a much faster, easier and extendable platform for RAG indexation.
Swiftide provides an asynchronous, parallel, streaming ingestion pipeline. To kick it off we have chunkers for markdown and code, metadata augmentation with QA, embedding with openai and storage in qdrant. More cool stuff will be added the upcoming weeks. And of course all of it in Rust.
Feedback is very welcome! Check it out at bosun-ai/swiftide