How Can We Update the Cognee Data's SYSTEM_ROOT_DIRECTORY and DATA_ROOT_DIRECTORY

[LStromann]

=we have implemented an cognee with help of s3 bucket now what we need is to we want to update the cognee system_root_directory and data_root_directory on each call because the cognee is unable to create an new instance.

This might be because it's similar to singleton. now the question is can we do a cognee related directories update on each call by the folders.

In Short Can we make it work for multiple users simultaneously. or it will create an new issue ?

This discussion was automatically pulled from Discord.

[LStromann]

hey <@423095375742631936> you can't update them directly but yes Cognee can work for multiple users simultaneously.
add this to your .env file - ENABLE_BACKEND_ACCESS_CONTROL=True

this gives each user their own private folder!
https://docs.cognee.ai/setup-configuration/relational-databases#backend-access-control

then set Up your S3 storage-
DATA_ROOT_DIRECTORY=s3://your-bucket/cognee/data SYSTEM_ROOT_DIRECTORY=s3://your-bucket/cognee/system

cognee automatically creates user-specific folders inside

[LStromann]

let us know if this works

[LStromann]

<@1373135538147692604> by multiple users i mean can we update the both directory on each request so it will be updated on every request or can we just create multiple instances ?

[LStromann]

yes you can update the directories on each call using the config -

cognee.config.data_root_directory(f"s3://bucket/user-{user_id}/data") cognee.config.system_root_directory(f"s3://bucket/user-{user_id}/system")

or you can try the single instance with isolation approach by setting ENABLE_BACKEND_ACCESS_CONTROL=true it automatically configures isolated storage paths for each users.

btw what's your use case?

[LStromann]

we are trying to isolate directory for organization_id and then project_id folders but as i can see we already use different dataset_name for different projects and my organization is asking it to update the above. now the case is if i done this i think it will be needs changed on each request like each cognee.add and each cognee.search functions which may conflict multiple requests done simultaneously. like if user a changes the env and user b also changed then whoever changed last is considered the directory and user a might be calling directory set by user b

[LStromann]

and we want to actually isolate on based on org id and project id we provide not the user id

[LStromann]

issue is not that it can be done or not but want to understand does it will create any new issue for live application like wrong directory check due use of asynchronous programming

[LStromann]

what do you think does this creates an new issue instead <@1373135538147692604>

[LStromann]

hey <@423095375742631936> yes isolating them can create issues if not implemented correctly but cognee's built-in multi-tenant system prevents these problems through async context isolation.

Cognee uses ContextVar to maintain separate configurations per async context when ENABLE_BACKEND_ACCESS_CONTROL=True

• each async request gets its own isolated context
• no shared state between concurrent requests

[LStromann]

so i just have to update the directories on every cognee.search and congnee.add and cognee.cognify functions right?

[LStromann]

doesn't it make slower ?

[LStromann]

no you don't need to manually update directories on every function call.
simply enable access control once then use the functions normally no manual directory updates needed. and it has has minimal performance impact. the context setup happens once per dataset operation, not per data item

[LStromann]

You can't just update directories on every cognee.search, cognee.add and cognee.cognify calls unfortunately. It's a global config for s3 storage

[LStromann]

you can have separate Cognee instances running to handle different s3 storages with their own config potentially

[LStromann]

but the documentation does not show that can we make multiple instances of cognee. in short it's like ```python3
import cognee
cognee.add(...)

[LStromann]

you can't make multiple instances of Cognee when writing python code in the SDK, but if you run it from different processes or run multiple Cognee backends you can route requests from customers to

If you need to separate them on a project/org basis on S3