SingleStore CEO sees little future for purpose-built vector databases

SingleStore is advancing its namesake database platform with a new release today that expands functionality for generative AI workloads, as well as both transactional and analytics data workloads.

The new SingleStore Pro Max database, which technically is also the SingleStore 8.5 release update, integrates new indexed vector search capabilities to help organizations build and support gen AI applications and retrieval augmented generation (RAG) use cases.

The indexed vector search update in SingleStore Pro Max does not mark the debut of vector capabilities in the SingleStore database. The database has had various forms of vector support since 2017, back when the company was known as MemSQL. The company rebranded as SingleStore in 2020, as it became a database that converges both Online Analytical Processing (OLAP) and Online Transaction Processing (OLTP) into a single unified platform.

As gen AI workloads have ramped up, so has the need for vector database capabilities, with native database platforms such as Pinecone emerging. In recent months, existing database vendors including DataStax, Neo4j, MongoDB, PostgreSQL and even Oracle have all added vector support. The ability to combine existing database capabilities with vectors is critical for organizations, according to SingleStore’s CEO, who believes that the narrow focus of just having a purpose-built vector-only database is not the right path forward.

“We provide you with a gen AI stack including vectors that allows you to build and model gen AI applications,” Raj Verma, CEO of SingleStore told VentureBeat. “What we believe is that a vector-only database is a feature set and not a database that is going to be around in probably two maximum three years, because it adds a further layer of complexity in your AI stack and what you want to have an effective gen AI stack is to take complexity out, not add further complexity.”

Vectors power hybrid search across structured and unstructured data

With support for both OLAP and OLTP workloads, SingleStore is sometimes referred to as a Hybrid Transactional and Analytical Processing (HTAP) database. Functionally what that means is the database can store, process and query multiple types of data, in different ways.

With the Pro Max release, SingleStore is bringing enhanced support for vector search across both structured and unstructured data. SingleStore has supported vector search since 2017 but the new release includes faster and more accurate algorithms like product quantization (PQ), Hierarchical Navigable Small World (HNSW) and Approximate Nearest Neighbor (ANN) vector indexing algorithms.

The enhanced index vector search promises that organizations will be able to benefit from all their data that is accessible from SingleStore for search as well for supporting gen AI applications.

Verma commented that having a database that only supports vectors and not other data types, can help an organization to quickly get into the gen AI space. However, it’s not a solution that considers the broader data landscape that most organizations inhabit.

“Just by adding the veneer of vectors, you aren’t going to hide the ugliness of the complexity of the data state that exists,” he said. 

Verma explained that SingleStore’s vision is that it can serve as a vector database option as part of a larger converged and simplified data estate that includes the other data an organization needs.

“Only simplicity scales and only through data consolidation can you get simplicity and speed which is required for a gen AI data estate to prosper,” he said.

Change Data Capture integrates Apache Iceberg

It is rare today for any organization to have all of its data in a single database.

What often happens is that data pipelines need to be created across different data repositories and applications. Inside a database, one of the most common ways that data is ingested from another data source is via Change Data Capture (CDC).

The SingleStore Pro Max includes enhanced change data capture capabilities to integrate data from MySQL and MongoDB databases as well Apache Iceberg based data lakes into a single database.

The support for Apache Iceberg is particularly noteworthy as it is an open-source data lake table format used by many leading vendors including IBM and Snowflake. Verma noted that SingleStore is strongly committed to partnerships with both IBM and Snowflake and having Iceberg support will go a long way to enabling easier integration.

“CDC capability allows our customers to have the ability to have the data from various sources brought into SingleStore which is extremely important for the entire retrieval, augment augmented generation workflow,” said Verma.