k8s deployment for neo4j

[jeanyu-habana]

Description

The purpose is to enable neo4j component to cover the installation and configuration in k8s env. This component will be a part of an end-to-end GenAI pipeline (named HybridRAG).

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

n/a

Tests

Tested the commands and configuration file (values.yaml) in k8s environment. Neo4j was properly installed and configured. Database was generated and can be accessed and updated from within the k8s environment, from command-line, and from the web console.

[letonghan]

Hi @jeanyu-habana, thanks for your contribution.
Please fix the ci test error of helm-test.

[jeanyu-habana]

Hi @jeanyu-habana, thanks for your contribution. Please fix the ci test error of helm-test.

fixed. @letonghan

[jeanyu-habana]

Can I have at least one more approval from the reviewers? @XinyaoWa @lvliang-intel

[yongfengdu]

You are using the neo4j helm chart from upstream, and have customized values.yaml file.
In this case, we don't have to put it into our k8s deployment directory. You can refer to how we use milvus in dataprep:

Include the dependency in your HybridRAG dependency:
https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/Chart.yaml#L20
Then customize your values in your HybridRAG values file:
https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/milvus-values.yaml#L4

[jeanyu-habana]

You are using the neo4j helm chart from upstream, and have customized values.yaml file. In this case, we don't have to put it into our k8s deployment directory. You can refer to how we use milvus in dataprep:

Include the dependency in your HybridRAG dependency: https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/Chart.yaml#L20 Then customize your values in your HybridRAG values file: https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/milvus-values.yaml#L4

Yes, this component can be used in our HybridRAG, but we have not decided if we will use neo4j or NebulaGraph in HybriRAG. The intent of this PR is to provide NebulaGraph as a general purpose microservice. HybridRAG is a work in progress, and we will reference the comment above as part of the HybridRAG dependency. However, in the context of this PR, do you think we can keep the content as is?
@yongfengdu

[yongfengdu]

@jeanyu-habana The purpose of deployment/kubernetes directory is to put values file for testing our own helm charts (https://github.com/opea-project/GenAIInfra/tree/main/helm-charts).
The current CI is also designed to fetch helm charts from here https://github.com/orgs/opea-project/packages.

Please take care of the CI test and maintain when necessary.

You are using the neo4j helm chart from upstream, and have customized values.yaml file. In this case, we don't have to put it into our k8s deployment directory. You can refer to how we use milvus in dataprep:
Include the dependency in your HybridRAG dependency: https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/Chart.yaml#L20 Then customize your values in your HybridRAG values file: https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/common/data-prep/milvus-values.yaml#L4

Yes, this component can be used in our HybridRAG, but we have not decided if we will use neo4j or NebulaGraph in HybriRAG. The intent of this PR is to provide NebulaGraph as a general purpose microservice. HybridRAG is a work in progress, and we will reference the comment above as part of the HybridRAG dependency. However, in the context of this PR, do you think we can keep the content as is? @yongfengdu

[jeanyu-habana]

@yongfengdu Thank you for meeting with me to discuss similar concerns over adding k8s components such as neo4j. Just to summarize, we went over the example in GenAIExamples and GenAIInfra (i.e., milvus), and I agreed when we implement our HybridRAG in GenAIExamples and GenAIInfra, we will follow the guidance. However, having neo4j k8s deployment in the GenAIComps repo still has value. The script here is tested and can be used to deploy neo4j as microservice in k8s. Please let me know if there is anything needed, and we will address them ASAP.

[xiguiw]

Yes, this component can be used in our HybridRAG, but we have not decided if we will use neo4j or NebulaGraph in HybriRAG. The intent of this PR is to provide NebulaGraph as a general purpose microservice.

This PR is NOT active for weeks.

@jeanyu-habana
Any progress about the HybriRAG? Or any components that use the neo4j?