Sharing configuration between different teams

[avnerstr]

I would like to know if it's possible to have different repos for different teams and a shared repo
so I will be able to have a shared repo that will contain a configuration that is relevant to all the teams

for example:
I have team-a and team-b, each one have it's own repo for their configurations but I want also to have a repo for shared configuration, so when team-a is asking for it's configuration it will get also the shared configuration in the shared repo.
for example, when I will access the spring cloud config via:
localhost:8888/team-a-app/default I will get also data from the https://git/common/config-repo.git

is it doable ?
spring:
cloud:
config:
server:
git:
uri: https://git/common/config-repo.git
repos:
team-a:
pattern: team-a-*
cloneOnStart: true
uri: https://git/team-a/config-repo.git
team-b:
pattern: team-b-*
cloneOnStart: false
uri: https://git/team-b/config-repo.git
team-c:
pattern: team-c-*
uri: https://git/team-a/config-repo.git

[ryanjbaxter]

I don't believe so but I can think of 2 possible solutions to maybe accomplish this another way.

1. Could you use a Git Submodule to point to the common repo from all the team repos?
2. You could setup 2 config servers, one for the common repo and one for the individual teams, then in spring.config.import you could list both config servers.

[avnerstr]

I tried your suggestion in option 1 and it seems that if there are updates in the submodule, they are not pulled to the config server. What do I miss here?

[ryanjbaxter]

Did you set

spring:
  cloud:
    config:
      server:
        git:
          cloneSubmodules: true

[avnerstr]

@ryanjbaxter, Yes I set this parameter.
without it, it wouldn't have clone the submodule.
But still it didn't get the changes, when I update the submodule.
You can try it yourself.
Unless I miss something, sounds like a bug.

[ryanjbaxter]

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

[avnerstr]

I can't provide an example with AWS code commit, but I will try provide an example with GitHub as the git source

[avnerstr]

hi @ryanjbaxter

the github project can be found here:
https://github.com/avnerstr/spring-cloud-config-server-example

There is a git repo for configuration
https://github.com/avnerstr/dev-configuration-repo and I configured another repo:
https://github.com/avnerstr/support-configuration-repo to be submodule of the first repo.

In order to see the issue, what seems like a bug:

1. Run the configuration server
2. open browser and run http://localhost:8888/springboot-no-ui-tests/stage
   you will see configuration information from all the repo (the main and the sub-module)
3. Change the text in configuration files in the submodule project https://github.com/avnerstr/support-configuration-repo
4. open browser and run http://localhost:8888/springboot-no-ui-tests/stage and you will see it didn't get the changes
5. Do the same on the main repo and you will see it does get the changes.

AFAIK the browser request is similar to the request the spring config client are asking, the problem will exists on the client as well.

Please update me with your results.

Thanks,
Avner

[ryanjbaxter]

I played around with the sample but I don't think we update submodules after they are cloned initially. I created an issue to track this
#1924

I guess that leaves us with option 2 from above to support what you were trying to do.

[avnerstr]

@ryanjbaxter eventually to solve this issue I will use the overide configuration - that is shared for all clients
I think that option 2 is abuse of a feature for failover. I don't think I would want that.
WDYT?

[ryanjbaxter]

The override feature is fine.

But I dont think option 2 is an abuse of a feature at all, I think its one of the benefits spring.config.import was trying to provide.

[avnerstr]

just to understand - you mean that I will do it in the client side, right?

I also found another solution that I can have repositories for support teams, repositories for developers and root repository to put configuration that is relevant to all teams:

  cloud:
    config:
      server:
        composite:
          # Support repositories
          - type: git
            uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/fallback-repo
            default-label: main
            force-pull: true
            username: XXX
            password: YYY
            clone-on-start: true
            repos:
              team1-support:
                clone-on-start: true
                pattern:
                  - 'springboot-no-ui-tests'
                clone-submodules: false
                uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/support
                force-pull: true
                default-label: main
                username: XXX
                password: YYY
          #Developers repositories
          -
            type: git
            uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/fallback-repo
            default-label: main
            force-pull: true
            username: XXX
            password: YYY
            clone-on-start: true
            repos:
              team1-dev:
                clone-on-start: true
                pattern:
                  - 'springboot-no-ui-tests'
                clone-submodules: false
                uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/dev1
                force-pull: true
                default-label: main
                username: XXX
                password: YYY
              team2-dev:
                clone-on-start: true
                pattern:
                  - 'team2-*'
                clone-submodules: false
                uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/dev2
                force-pull: true
                default-label: main
                username: XXX
                password: YYY
          #Top Level Configuration (shared between all teams)
          - type: git
            uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/mgmt
            default-label: main
            force-pull: true
            username: XXX
            password: YYY
            clone-on-start: true

[ryanjbaxter]

you mean that I will do it in the client side, right?

Correct

[avnerstr]

The problem with that,
that I don't see from the documentation that spring.config.import can support security
Unlike the the old way in the bootstrap

spring:
  application:
    name: springboot-no-ui-tests
  cloud:
    config:
      uri: http://localhost:8888/

especially when I need to use CustomConfigServiceBootstrapConfiguration because I want to read the username/password at runtime. I don't think it's possible in the new way.
Correct me if I'm wrong

[ryanjbaxter]

We still support security using spring.conig.import the config client still takes that into account. However since its not happening during bootstrap that code might need to live elsewhere. I am not sure where off the top of my head.

[avnerstr]

I think that I tried to make the CustomConfigServiceBootstrapConfiguration works with the new way and it didn't work.
But I will have to give it another try to be sure.
I'm talking about the : Providing A Custom RestTemplate, if that wasn't clear.
If that won't work, I will have to continue to use the bootstrap way.

[ryanjbaxter]

This is probably the class you want to look at registering as a bean and customize it
https://github.com/spring-cloud/spring-cloud-config/blob/main/spring-cloud-config-client/src/main/java/org/springframework/cloud/config/client/ConfigClientRequestTemplateFactory.java

Anyways this is a different topic to your original one. I think you have a workaround for what you want to do and have a few other options to try. Can we close this issue?

[avnerstr]

Hi

Yes you can close this issue
thanks for all the help !