Allow multiple sopsFiles for easier common secrets sharing across multiple configuration by vdbe · Pull Request #417 · Mic92/sops-nix

vdbe · 2023-10-08T08:13:19Z

Allow multiple sopsFiles for easier secret sharing across multiple configurations.

For example you have files

secrets/common.yaml

hashed_password: ....
vpn_user: ...
vpn_password: ...
vpn_cert: | ...

secrets/<system>/default.yaml

vpn_user: ...
vpn_password: ...

with sops.defaultSopsFiles = [ ./secrets/common.yaml ./secrets/<system>/default.yaml ]; in the configuration for <system>.

This would allow overriding of shared secrets across hosts without manually checking and changing the required sopsFile per secret.

vdbe · 2023-10-08T08:14:36Z

Current changes are just a rough draft

Mic92 · 2023-10-08T09:52:15Z

Only question is how you order priority between those in case keys overlap? Otherwise it would make sense to throw an error so that users don't accidentally shadow secrets.

vdbe · 2023-10-08T10:03:54Z

My idea was to have the files in increasing priority,
this way you can have a default common file and append system/user specific files.

In other words search the list of sops files in reverse order and use the first found secret.

In the example above vpn_user and vpn_password in secrets/common.yaml would be shadowed by vpn_user and vpn_password in secrets/<system>/default.yaml.

Mic92 · 2023-10-08T10:19:11Z

It's just hard to enforce an order in lists if you those lists get defined in different nixos modules.

Mic92 · 2023-10-08T10:22:03Z

Ah. I think one can use lib.mkBefore for host files.

Mic92 · 2023-10-08T10:23:31Z

+          Sops file the secret is loaded from.
+        '';
+      };
      sopsFileHash = mkOption {


We should deprecated this one than.

Deprecate sopsFiles? Just added that this PR, do you mean deprecate sopsFile?

We shouldn't really need both sopsFileHash and sopsFilesHash, no?

They could be point to the same value.

Mic92 · 2023-10-08T10:23:56Z

@@ -172,6 +189,13 @@ in {
        Default sops file used for all secrets.
      '';
    };


And this one with a warning.

vdbe

Something like this?

vdbe · 2023-10-14T10:58:15Z

-      sopsFile = lib.mkOptionDefault cfg.defaultSopsFile;
-      sopsFileHash = mkOptionDefault (optionalString cfg.validateSopsFiles "${builtins.hashFile "sha256" config.sopsFile}");
+      sopsFile = mkOptionDefault cfg.defaultSopsFile;
+      sopsFiles = if options.sopsFile.isDefined then warn "`sops.secrets.<name>.sopsFile` is being deprecated, use `sops.secrets.<name>.sopsFiles` instead" [ config.sopsFile ] else (lib.mkOptionDefault cfg.defaultSopsFiles);


defaultSopsFile/sopsFile takes priority over defaultSopsFiles

Can we not add defaultSopsFile to the new defaultSopsFiles list?

Would it not be better to keep defaultSopsFile and defaultSopsFiles separate and place sopsFile before sopsFiles per secret?
If you merge in default you would need to remove sopsFile
otherwise sopsFiles would become [ sopsFile defaultSopsFile ] ++ defaultSopsFiles.

The only way I was able to implement this feels however a bit hacky

config = mkMerge [{ sopsFile = mkOptionDefault cfg.defaultSopsFile; sopsFiles = mkOptionDefault cfg.defaultSopsFiles; sopsFilesHash = mkOptionDefault (optionals cfg.validateSopsFiles (forEach config.sopsFiles (builtins.hashFile "sha256"))); } { sopsFiles = mkIf (config.sopsFile != null) (mkOverride options.sopsFile.highestPrio (mkBefore [config.sopsFile])); }];

The effect of this is that when both sopsFile and sopsFiles are defined (and not null) with different priorities only one will make it into config.sopsFiles

sopsFile or sopsFiles also take priority over defaultSopsFile and defaultSopsFiles

vdbe · 2023-10-14T11:00:49Z

+	for i := len(files) - 1; i >= 0; i-- {
+		err := app.validateSopsFile(secret, &files[i])
+		if err == nil {
+			// Found valid sopsFile
+			break
+		} else if i == 0 {
+			// No valid sopsFile found in sopsFiles
+			return err
+		}
+	}


The error "secret %s defined the format of %s as %s, but it was specified as %s in %s before" from validateSopsFile is ignored

Mhm. I would build a new error message that would give you all files where the secret has been looked for instead.

vdbe · 2023-10-14T11:06:02Z

+				return fmt.Errorf("Secret of type %s in %s is not supported", s.Format, s.SopsFiles[i])
+			}
+		}
 		switch s.Format {


Assuming all sopsFiles have the same format

Mic92 · 2023-10-24T13:40:35Z

 }

-func (app *appContext) loadSopsFile(s *secret) (*secretFile, error) {
+func (app *appContext) loadSopsFile(s *secret, sopsFileIndex int) (*secretFile, error) {


Can we not pass in the sopsFile as a pointer here rather than using an index?

Mic92 · 2023-10-24T13:41:10Z

-		if err != nil {
-			return err
+	files := []secretFile {};
+	for index, sopsFile := range secret.SopsFiles {


You already have you reference here.

gabevenberg · 2024-05-31T23:02:14Z

is this dead in the water? is there another way to share secrets among hosts without copying secrets between individual hosts secrets.yaml files?

bmwagner18 · 2026-02-07T04:43:30Z

Mic92's comment on this issue solved my issue:

#378 (comment)

let
  wordPressSecrets.sopsFile = ../../secrets/wordpress.yaml;
in
  # create the sops secrets paths so I can reference them
  sops.secrets."wordpress.db.host" = wordPressSecrets;
  sops.secrets."wordpress.db.user" = wordPressSecrets;
  
  sops.secrets."wordpress.db.password" = wordPressSecrets // {
    # here you can override the default
  };
};

Mic92 reviewed Oct 8, 2023

View reviewed changes

vdbe commented Oct 14, 2023

View reviewed changes

Mic92 reviewed Oct 24, 2023

View reviewed changes

vdbe force-pushed the master branch from addbe58 to e9ff03f Compare November 12, 2023 18:38

vdbe marked this pull request as ready for review November 12, 2023 18:55

vdbe requested a review from Mic92 December 3, 2023 10:35

vdbe added 4 commits February 2, 2024 13:33

feat(shadowing): golang

262a222

feat(shadowing): nixos module

8ee9a71

feat(shadowing): home manager module

0068d61

test(shadowing): sops-install-secrets

c623385

vdbe force-pushed the master branch from 707e856 to c623385 Compare February 2, 2024 12:51

fabiosouzadev approved these changes Aug 14, 2024

View reviewed changes

bjornfor mentioned this pull request Sep 16, 2024

How to take secrets from multiple files? #622

Open

Conversation

vdbe commented Oct 8, 2023

Uh oh!

vdbe commented Oct 8, 2023

Uh oh!

Mic92 commented Oct 8, 2023

Uh oh!

vdbe commented Oct 8, 2023

Uh oh!

Mic92 commented Oct 8, 2023

Uh oh!

Mic92 commented Oct 8, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vdbe left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vdbe Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gabevenberg commented May 31, 2024

Uh oh!

bmwagner18 commented Feb 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

vdbe Oct 28, 2023 •

edited

Loading