- [8 - 40X faster inference than HTTP based model serving](https://postgresml.org/blog/postgresml-is-8x-faster-than-python-http-microservices)

- [Millions of transactions per second](https://postgresml.org/blog/scaling-postgresml-to-one-million-requests-per-second)

- [Horizontal scalability](https://github.com/postgresml/pgcat)

There are two stages in building a knowledge based chatbot:

- Build a knowledge base by ingesting documents, chunking documents, generating embeddings and indexing these embeddings for fast query

This tool automates the above two stages and provides a command line interface to build and deploy a knowledge based chatbot.

* [Introducing the OpenAI Switch Kit: Move from closed to open-source AI in minutes](introducing-the-openai-switch-kit-move-from-closed-to-open-source-ai-in-minutes.md)

* [Speeding up vector recall 5x with HNSW](speeding-up-vector-recall-5x-with-hnsw.md)

* [How-to Improve Search Results with Machine Learning](how-to-improve-search-results-with-machine-learning.md)

* [The 1.0 SDK is Here](the-1.0-sdk-is-here.md)

* [Using PostgresML with Django and embedding search](using-postgresml-with-django-and-embedding-search.md)

* [pgml-chat: A command-line tool for deploying low-latency knowledge-based chatbots](pgml-chat-a-command-line-tool-for-deploying-low-latency-knowledge-based-chatbots-part-i.md)

* [Announcing Support for AWS us-east-1 Region](announcing-support-for-aws-us-east-1-region.md)

* [LLM based pipelines with PostgresML and dbt (data build tool)](llm-based-pipelines-with-postgresml-and-dbt-data-build-tool.md)

<figure><img src=".gitbook/assets/image (8).png" alt=""><figcaption></figcaption></figure>

<figure><img src=".gitbook/assets/image (9).png" alt=""><figcaption></figcaption></figure>

To take advantage of latency savings, you can [deploy a dedicated PostgresML database](https://postgresml.org/signup) in `us-east-1` today. We make it as simple as filling out a very short form and clicking "Create database".

April 21, 2023

This article is the first in a multipart series that will show you how to build a post-modern semantic search and recommendation engine, including personalization, using open source models.

| GPU | 17ms | $72 | 6 hours |

| OpenAI | 300ms | $299 | millennia |

You can also find embedding models that outperform OpenAI's `text-embedding-ada-002` model across many different tests on the [leaderboard](https://huggingface.co/spaces/mteb/leaderboard). It's always best to do your own benchmarking with your data, models, and hardware to find the best fit for your use case.

On the bright side, the open-source realm has emerged as a potent contender, not just in reaction to OpenAI's shortcomings but as a genuine advancement in its own right. We're all about making the benefits of open-source models accessible to as many folks as possible. So, we've made switching from OpenAI to open-source as easy as possible with a drop-in replacement. It lets users specify any model they’d like in just a few lines of code. We call it the OpenAI Switch Kit. Read on to learn more about why we think you’ll like it, or just try it now and see what you think.

We think so. Open-source models have made remarkable strides, not only catching up to proprietary counterparts but also surpassing them across multiple domains. The advantages are clear:

| Full Text Search | - | ✅ |

| Geospatial Search | - | ✅ |

Both MindsDB and PostgresML support many classical machine learning algorithms to do classification and regression. They are both able to load ~~the latest LLMs~~ some models from Hugging Face, supported by underlying implementations in libtorch. I had to cross that out after exploring all the caveats in the MindsDB implementations. PostgresML supports the models released immediately as long as underlying dependencies are met. MindsDB has to release an update to support any new models, and their current model support is extremely limited. New algorithms, tasks, and models are constantly released, so it's worth checking the documentation for the latest list.

Another difference is that PostgresML also supports embedding models, and closely integrates them with vector search inside the database, which is well beyond the scope of MindsDB, since it's not a database at all. PostgresML has direct access to all the functionality provided by other Postgres extensions, like vector indexes from [pgvector](https://github.com/pgvector/pgvector) to perform efficient KNN & ANN vector recall, or [PostGIS](http://postgis.net/) for geospatial information as well as built in full text search. Multiple algorithms and extensions can be combined in compound queries to build state-of-the-art systems, like search and recommendations or fraud detection that generate an end to end result with a single query, something that might take a dozen different machine learning models and microservices in a more traditional architecture.

| translation\_en\_to\_es | t5-base | 1573 | 1148 | 294 |

| summarization | sshleifer/distilbart-cnn-12-6 | 4289 | 3450 | 479 |

There is a general trend, the larger and slower the model is, the more work is spent inside libtorch, the less the performance of the rest matters, but for interactive models and use cases there is a significant difference. We've tried to cover the most generous use case we could between these two. If we were to compare XGBoost or other classical algorithms, that can have sub millisecond prediction times in PostgresML, the 20ms Python service overhead of MindsDB just to parse the incoming query would be hundreds of times slower.

This article is the third in a multipart series that will show you how to build a post-modern semantic search and recommendation engine, including personalization, using open source models. You may want to start with the previous articles in the series if you aren't familiar with PostgresML's capabilities.

<figure><img src=".gitbook/assets/image (24).png" alt=""><figcaption><p>Embeddings can be combined into personalized perspectives when stored as vectors in the database.</p></figcaption></figure>

!!! note