Navigating Multi-Tenancy Challenges with Bedrock KnowledgeBase

In the ever-evolving world of cloud-based applications, understanding and adapting to service limitations is crucial. Recently, while working on a multi-tenant application using Bedrock KnowledgeBase (KB), I encountered a challenge that required a reevaluation of our initial design.

The Challenge: Limit on Knowledge Bases per Account

Bedrock KnowledgeBase is an orchestrator for implementing Language Model RAG, handling the vectorization of files into vector stores and generating context-aware conversations when combined with Bedrock Agent. In our application, we were using OpenSearch as our vector store, creating separate KBs and indices for each tenant to ensure data isolation. However, a quota on the maximum number of KBs per account posed a limitation, restricting our application to support only up to 100 tenants.

The Solution: Sharing Knowledge Bases across Multiple Tenants

To overcome this challenge, we modified our design so that KBs and indices are shared across multiple tenants. To avoid tenant data being referenced during conversations with other tenants, we adopted a method of attaching custom metadata (tenant_id) to each document and filtering documents by that ID during retrieval. This approach allowed us to build the application while successfully avoiding the constraints.

Attaching Custom Metadata

To attach metadata to documents, we used the following code:

  ingest_documents_response = client.ingest_knowledge_base_documents(knowledgeBaseId='string', dataSourceId='string', clientToken='string', documents=[{'metadata': {'type': "IN_LINE_ATTRIBUTE", 'inlineAttributes': [{'key': 'tenant_id', 'value': {'type': "STRING", 'stringValue': "$tenant_id", }}]}, 'content': {...}}])  

Filtering Documents by Metadata

To filter documents by metadata when conversing with the agent, we implemented the following code:

  invoke_agent_response = boto3.client.invoke_agent('knowledgeBaseConfigurations': [{"knowledgeBaseId": "$vector_store_id", "description": "Knowledge base for document retrieval", "retrievalConfiguration": {"vectorSearchConfiguration": {"filter": {"equals": {"key": "tenant_id", "value": "$tenant_id"}}}}}]])  

Future Plans: Monitoring Data Isolation

With the current implementation, we can now build the application while ensuring data isolation between tenants. To further secure the system, I am considering creating a monitoring mechanism to ensure that one tenant cannot access another tenant's data. For example, I am thinking of creating multiple test tenants, inserting different documents into the same vector store for each, asking questions about other tenants' documents, and verifying that no answers are returned. This script could be executed regularly in staging environments.

Relevance to North East Region and India

As cloud-based applications become increasingly popular, understanding and adapting to service limitations is essential for developers working in the North East region and across India. The challenges and solutions outlined in this article can serve as a valuable resource for developers working with Bedrock KnowledgeBase and other cloud services.

Reflections and Forward-Looking Thoughts

This experience has underscored the importance of checking cloud service specifications before deciding on system design. By adapting our design to accommodate the quota on Knowledge Bases per account, we were able to build a more efficient and scalable multi-tenant application. As developers continue to push the boundaries of what's possible with cloud-based services, it will be essential to remain vigilant about service limitations and adapt our designs accordingly.