Skip to content

Unions nested inside structures are not parsed #37

@RduMarais

Description

@RduMarais

Hello everyone!
Thank you for maintaining this library, and fixing various bugs with the original library.
I am trying to parse headers with a anonymous union defined inside a struct :

typedef struct
{
  volatile union
  {
    volatile uint8_t u8;
    volatile uint16_t u16;
    volatile uint32_t u32;
  } PORT [32];
       uint32_t RESERVED0[864];
  volatile uint32_t TER;
   .....
} ITM_Type;

However, this union is not parsed as such, and i have no way of isolating the data allocated inside the union from the rest of the structure.
Output, printed as another C header gives :

typedef struct ITM_Type
{
  volatile uint8_t u8;
  volatile uint16_t u16;
  volatile uint32_t u32;
  PORT ;
  uint32_t RESERVED0;
  volatile uint32_t TER;
   .....
} ITM_Type;

As you can see the resulting memory allocated is not correct.

The output in json is :

{
    "headerFileName": "report_issue/input_union.h",
    "curClass": "",
    "classes": {
        "ITM_Type": {
            "nested_classes": [],
            "abstract": false,
            "final": false,
            "namespace": "",
            "using": {},
            "name": "ITM_Type",
            "bare_name": "<anon-struct-1>",
            "inherits": [],
            "filename": "report_issue/input_union.h",
            "line_number": 2,
            "methods": {
                "public": [],
                "protected": [],
                "private": []
            },
            "properties": {
                "public": [
                    {
                        "extern": false,
                        "array": 0,
                        "filename": "report_issue/input_union.h",
                        "line_number": 6,
                        "function_pointer": 0,
                        "type": "volatile uint8_t",
                        "name": "u8",
                        "aliases": [
                            "volatile"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "volatile uint8_t"
                    },
                    {
                        "extern": false,
                        "array": 0,
                        "filename": "report_issue/input_union.h",
                        "line_number": 7,
                        "function_pointer": 0,
                        "type": "volatile uint16_t",
                        "name": "u16",
                        "aliases": [
                            "volatile"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "volatile uint16_t"
                    },
                    {
                        "extern": false,
                        "array": 0,
                        "filename": "report_issue/input_union.h",
                        "line_number": 8,
                        "function_pointer": 0,
                        "type": "volatile uint32_t",
                        "name": "u32",
                        "aliases": [
                            "volatile"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "volatile uint32_t"
                    },
                    {
                        "extern": false,
                        "array_size": "32",
                        "array": 1,
                        "filename": "report_issue/input_union.h",
                        "line_number": 9,
                        "function_pointer": 0,
                        "type": "PORT",
                        "name": "",
                        "aliases": [
                            "PORT"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "PORT"
                    },
                    {
                        "extern": false,
                        "array_size": "864",
                        "array": 1,
                        "filename": "report_issue/input_union.h",
                        "line_number": 10,
                        "function_pointer": 0,
                        "type": "uint32_t",
                        "name": "RESERVED0",
                        "aliases": [
                            "uint32_t"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "uint32_t"
                    },
                    {
                        "extern": false,
                        "array": 0,
                        "filename": "report_issue/input_union.h",
                        "line_number": 11,
                        "function_pointer": 0,
                        "type": "volatile uint32_t",
                        "name": "TER",
                        "aliases": [
                            "volatile"
                        ],
                        "typedef": null,
                        "constant": 0,
                        "constexpr": 0,
                        "reference": 0,
                        "pointer": 0,
                        "static": 0,
                        "typedefs": 0,
                        "class": 0,
                        "fundamental": 0,
                        "unresolved": true,
                        "namespace": "",
                        "property_of_class": "<anon-struct-1>",
                        "mutable": false,
                        "raw_type": "volatile uint32_t"
                    },
                ],
                "protected": [],
                "private": []
            },
            "enums": {
                "public": [],
                "protected": [],
                "private": []
            },
            "typedefs": {
                "public": [],
                "protected": [],
                "private": []
            },
            "forward_declares": {
                "public": [],
                "protected": [],
                "private": []
            },
            "declaration_method": "struct"
        }
    },
    "functions": [],
    "pragmas": [],
    "defines": [],
    "includes": [],
    "headerFileNames": [
        "report_issue/input_union.h"
    ],
    "enums": [],
    "variables": [],
    "global_enums": {},
    "typedefs": {},
    "namespaces": [],
    "using": {}
}

Thanks in advance for any help

Activity

changed the title [-]Unions nested inside structures are not parsed as unions[/-] [+]Unions nested inside structures are not parsed[/+] on Feb 3, 2020
virtuald

virtuald commented on Feb 3, 2020

@virtuald
Member

Definitely a bug. I don't really have an interest in fixing this myself (lots of stuff going on), but I can provide guidance for debugging/fixing this if you would like to fix it.

RduMarais

RduMarais commented on Feb 4, 2020

@RduMarais
Author

I had a lookat the parsing method, it appears that the method evaluate_class_stack expects only one keyword in the declaration (in my case it was not recognized because the union is volatile.
I can try to change this behavior and use if "struct" in self.nameStack etc in :

 def _evaluate_class_stack(self):
        parent = self.curClass
        if parent:
            debug_print("found nested subclass")
            self.accessSpecifierStack.append(self.curAccessSpecifier)
        if self.nameStack[0] == "typedef":
            del self.nameStack[0]
        if len(self.nameStack) == 1:
            if self.nameStack[0] == "struct":
                self.anon_struct_counter += 1
                # We cant handle more than 1 anonymous struct, so name them uniquely
                self.nameStack.append("anon-struct-%d" % self.anon_struct_counter)
            elif self.nameStack[0] == "union":
                self.anon_union_counter += 1
                # We cant handle more than 1 anonymous union, so name them uniquely
                self.nameStack.append("anon-union-%d" % self.anon_union_counter)

However, when I try with only a union declarator, the class is parsed, but not its instance in the structure. In my case, I have an array of unions, and the parser does not see it.

virtuald

virtuald commented on Feb 4, 2020

@virtuald
Member

I find that setting the environment variable CPPHEADERPARSER_DEBUG=1 is really useful for figuring out what the parser is doing.

RduMarais

RduMarais commented on Feb 4, 2020

@RduMarais
Author

I used this to investigate a little bit, this is my debug log :
for my array

[3192] <anon-struct-10> (public) 
[3228] trace (Empty Stack)
[3181] Evaluating stack ['PORT', '[', '32', ']']
       BraceDepth: 1 (called from 2952)
[3192] <anon-struct-10> (public) 
[3312] trace
[2356] trace
[1186] var trace ['PORT', '[', '32', ']']
[1209] Array
[1221] Variable: ['PORT']

for a nested structure in the same file

[3192] <anon-union-1>::<anon-struct-2> (public) 
[3228] trace (Empty Stack)
[3181] Evaluating stack ['b']
       BraceDepth: 1 (called from 2952)
[3192] <anon-union-1> (public) 
[3312] trace
[2356] trace
[2383] DEANONYMOIZING b to type '<anon-struct-2>'
[1186] var trace ['<anon-struct-2>', 'b']
[1221] Variable: ['<anon-struct-2>', 'b']
[3181] Evaluating stack ['uint32_t', 'w']
       BraceDepth: 1 (called from 2952)

from my comprehension, the parser gets to the line 2371, where it decides what to do with the variable.

  • if the variable declaration has a length of 1, it checks for nested class types.
  • if not, it just declares a variable, assuming that the first keyword is the type.
    Hence it goes to the declaration of a CppVariableand sets the flags for an array of variables.

So my second issue here is that i can not parse an array of nested classes.
I don't know if just changing the condition to ( len(self.nameStack) == 1 or self.nameStack[1]=="[") would correct this or if it breaks a few things in the library. I try the 2 modifications and make a few checks.

RduMarais

RduMarais commented on Feb 4, 2020

@RduMarais
Author

So I made a pull request with a few changes that should not break anything.

This bug is made of 2 problems :

  1. The parser does not parse nested classes with preprocessor keywords
  2. The parser does not parse arrays of nested classes

the second issue should be adressed in the PR but the first one needs deeper changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Unions nested inside structures are not parsed · Issue #37 · robotpy/robotpy-cppheaderparser