Cannot parse partial configs

[kenneth-vkd]

Describe the bug
Unable to process config file with http block

To Reproduce
Parse this file:
template-config-no-http.txt

It works if you wrap it in an http block:
template-config-http.txt

Expected behavior
Since the file is a valid partial configuration, it would be expected to have this work

Your environment

• Operating System Ubuntu 20.04
• Version of crossplane crossplane 0.5.7

[rogueloop]

the same issue with server block - parsing returns lots of errors
got some alternative to this? @kenneth-vkd

[kenneth-vkd]

@rogueloop have not yet found another solution.
Currently reading a template config file to a string and the replace the simple parts, such as server_name and ssl certificate paths.
For more "complex" things, such as adding directives at specific points or remove directives, I convert the string to a list with .splitlines() and then use .join() to merge it back to a single string before creating a file

[astrowonk]

Yeah I want to edit the file(s) I have in sites-available and since they are partial (just a server block) that gets pulled into the main config with an include, clossplane throws nothing but errors. Kind of defeats the purpose ...

I could use the combine param on the nginix.conf file but then there's no way to split out just what's in sites-available.

Isn't putting server blocks insites-available a best practice? why wouldn't crossplane support editing those files?

[nikhilweee]

Same issue here. I wish to edit just the files in sites-available, I was hoping there was a way to do so.

[aliqandil]

I do have a hack handy, I may later turn it into a full fledged python library, but here is the core idea:

Instead of the default lex function, I use this one:

import crossplane
def lex( self, filename = None ):
		it = ( line + '\n' for line in self.template.split("\n") )
		it = crossplane.lexer._lex_file_object(it)
		it = crossplane.lexer._balance_braces(it)
		for token, line, quoted in it:
			yield (token, line, quoted) 

Where self.template is my sites-available config as a string, then using mocking, I patch the code like this:

from unittest.mock import patch
with patch('crossplane.parser.lex', lex):
			payload = crossplane.parse(f"{self.template_name}.conf",
				comments=True,
				single=True,
				check_ctx=False,
				check_args=False
			)

This way I can fool the library that it's reading a partial config from a file named f"{self.template_name}.conf", without actually needing to write my config strings into a file and making sure they make sense in the context of the global nginx.conf file.

[k0pernikus]

I have the same use case using crossplane@0.5.8:

I only want to edit a single file in my /etc/nginx/sites-available/MY_SITE, yet I would have to parse the entire config:

crossplane parse /etc/nginx/nginx.conf --indent=4 > all_files.json

and

crossplane build all_files.json

would then overwrite overt 20 files.

I want to make a single, atomic, edit in only one file (/etc/nginx/sites-available/MY_SITE) and only that one should be overwritten.

[k0pernikus]

I want to make a single, atomic, edit in only one file (/etc/nginx/sites-available/MY_SITE) and only that one should be overwritten.

I ditched crossplane and went with this quick and dirty solution in the scope of a bash / sh script:

APPEND_AFTER_LINE=$(grep -n -m 1 server /etc/nginx/sites-enabled/MY_SITE | grep -Po '^[0-9]+') && \
    sed -i "$APPEND_AFTER_LINE a listen 443 ssl;\nssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;\nssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;" /etc/nginx/sites-enabled/MY_SITE

This is a very naive script, as it just dumps text after the first line containing server, yet it works for my use case good enough. The grep commands find the first line where the searched string comes up, and via sed, I append it after the file number. Since I wanted to add multiple lines, I separate the lines via \n. Note: The added chunk is not indented as the rest of the file.

[belal-i]

I'm facing the same issue. I have to load an Nginx config file from a template, manipulate it internally in my code, and write it back into a container environment. The output file needs to have server as its outermost block, but in order for crossplane to parse my template, that block needs to be wrapped in an http block, which poses a bit of a challenge.

However, I have a workaround for this.

Let's say I have a base-nginx.conf like this:

http {
  server {
    listen 80;
    listen [::]:80;
    location / {
      return 200 "OK";
      add_header Content-Type text/plain;
    }
    server_name myhost.com;
    location ~ /.well-known/ {
        allow all;
        root /var/www/html;
    }
  }
}

Then I can do this:

import crossplane

base_template = crossplane.parse('base-nginx.conf')
# not elegant, but works
nginx_conf = base_template['config'][0]['parsed'][0]['block']


# Functions to manipulate nginx_conf as needed...

# Write nginx_conf - without the external http block - to a working nginx.conf
with open('nginx.conf', 'w') as fh:
    fh.write(crossplane.build(nginx_conf))

And this results in the following, which works for me:

server {
  listen 80;
  listen [::]:80;
  location / {
    return 200 "OK";
    add_header Content-Type text/plain;
  }
  server_name myhost.com;
  location ~ /.well-known/ {
      allow all;
      root /var/www/html;
  }
}

It's not so pretty to do base_template['config'][0]['parsed'][0]['block'] but my template is just a static file, so I don't see what could go wrong. Ofc it would be great if there were a more elegant way to do this, but for now it works... or am I doing something wrong?

Also, it's interesting how crossplane.parse() complains about the missing outer http block, but crossplane.build() doesn't mind.