[lldb][framework] Glob headers from source for framework

[chelcassanova]

When gathering the headers to fix up and place in LLDB.framework, we were previously globbing the header files from a location in the build directory. This commit changes this to glob from the source directory instead, as we were globbing from the build directory without ensuring that the necessary files were actually in that location before globbing.

[llvmbot]

@llvm/pr-subscribers-lldb

Author: Chelsea Cassanova (chelcassanova)

Changes

When gathering the headers to fix up and place in LLDB.framework, we were previously globbing the header files from a location in the build directory. This commit changes this to glob from the source directory instead, as we were globbing from the build directory without ensuring that the necessary files were actually in that location before globbing.

---

Full diff: https://github.com/llvm/llvm-project/pull/148736.diff

1 Files Affected:

• (modified) lldb/cmake/modules/LLDBFramework.cmake (+16-10)

diff --git a/lldb/cmake/modules/LLDBFramework.cmake b/lldb/cmake/modules/LLDBFramework.cmake
index bbd717a982cf3..adc50e493bc48 100644
--- a/lldb/cmake/modules/LLDBFramework.cmake
+++ b/lldb/cmake/modules/LLDBFramework.cmake
@@ -70,15 +70,21 @@ endif()
 
 find_program(unifdef_EXECUTABLE unifdef)
 
-# All necessary header files will be staged in the include directory in the build directory,
-# so just copy the files from there into the framework's staging directory.
-set(lldb_build_dir_header_staging "${CMAKE_BINARY_DIR}/include/lldb")
-set(lldb_framework_header_staging "${CMAKE_CURRENT_BINARY_DIR}/FrameworkHeaders")
-file(GLOB lldb_build_dir_header_staging_list ${lldb_build_dir_header_staging}/*)
-foreach(header ${lldb_build_dir_header_staging_list})
+# Glob all necessary header files from source and place them into a list.
+# These headers will then be collected for the framework and placed in the
+# FrameworkHeaders staging directory.
+file(GLOB public_headers ${LLDB_SOURCE_DIR}/include/lldb/API/*.h)
+set(generated_public_headers ${LLDB_OBJ_DIR}/include/lldb/API/SBLanguages.h)
+file(GLOB root_public_headers ${LLDB_SOURCE_DIR}/include/lldb/lldb-*.h)
+file(GLOB root_private_headers ${LLDB_SOURCE_DIR}/include/lldb/lldb-private*.h)
+list(REMOVE_ITEM root_public_headers ${root_private_headers})
+set(lldb_framework_header_staging_list ${public_headers} ${generated_public_headers} ${root_public_headers})
+
+set(lldb_framework_header_staging_dir ${CMAKE_CURRENT_BINARY_DIR}/FrameworkHeaders)
+foreach(header ${lldb_framework_header_staging_list})
 
   get_filename_component(basename ${header} NAME)
-  set(staged_header ${lldb_framework_header_staging}/${basename})
+  set(staged_header ${lldb_framework_header_staging_dir}/${basename})
 
   if(unifdef_EXECUTABLE)
     # unifdef returns 0 when the file is unchanged and 1 if something was changed.
@@ -106,9 +112,9 @@ add_dependencies(liblldb-resource-headers liblldb-header-staging)
 add_dependencies(liblldb liblldb-resource-headers)
 
 # Take the headers from the staging directory and fix up their includes for the framework.
-# Then write them to the output directory.
+# Then write them to the framework itself.
 # Also, run unifdef to remove any specified guards from the header files.
-file(GLOB lldb_framework_header_staging_list ${lldb_framework_header_staging}/*)
+file(GLOB lldb_framework_staged_headers ${CMAKE_CURRENT_BINARY_DIR}/FrameworkHeaders/*.h)
 foreach(header ${lldb_framework_header_staging_list})
 
   set(input_header ${header})
@@ -116,7 +122,7 @@ foreach(header ${lldb_framework_header_staging_list})
   set(output_header $<TARGET_FILE_DIR:liblldb>/Headers/${header_basename})
 
   add_custom_command(TARGET liblldb POST_BUILD
-    COMMAND ${LLDB_SOURCE_DIR}/scripts/framework-header-fix.py -f lldb_main -i ${input_header} -o ${output_header} -p ${unifdef_EXECUTABLE} USWIG
+    COMMAND "${Python3_EXECUTABLE}" ${LLDB_SOURCE_DIR}/scripts/framework-header-fix.py -f lldb_main -i ${input_header} -o ${output_header} -p ${unifdef_EXECUTABLE} USWIG
     COMMENT "LLDB.framework: Fix up and copy framework headers"
   )
 endforeach()

[JDevlieghere]

This is still globbing the build directory, so this patch still has the issue from the description. I imagine you want to do something like this instead:

1. Glob the source directory and collect all the header paths.
2. Iterate through all the header paths from the source directory and create a target for each of them to stage it. The target should have a file dependency on the header from the source directory and be a dependency of a top level staging target so you can do ninja lldb-stage-headers or something to verify this works. You probably want to make this target a dependency of liblldb. While doing this, you're going to iterate over the list of headers, so now's a good time to extract the basename and created a list with names of headers.
3. Iterate through the list of header filenames and create a target for each of them to fix them up for the framework. Make every target depend on its staging counterpart and create a top level target so you can do ninja lldb-fixup-headers or something. You probably want to make this a dependency of liblldb as well.

As I'm writing this, why do you need to do to the staging at all? I thought the staging was to put the headers in <build dir>/include/lldb but this code lives in LLDBFramework so it's for the framework only, and the headers are staged in FrameworkHeaders. Do we need this at all? The framework-header-fix script is already doin the unifdefing so I'm not sure what this is doing in the first place. If that's the case, then the code that globs the source dir and puts the headers in the build dir should take the role of (1).

[chelcassanova]

Do we need this at all? The framework-header-fix script is already doin the unifdefing so I'm not sure what this is doing in the first place.

If by this you mean FrameworkHeaders then I would say that I like having a staging directory to work from for the framework headers specifically, though I do agree that since the framework fix script already unifdefs the headers then the unifdefing that we have before we run the script should not be necessary anymore. I think from there then that loop that we have for unifdefing can be used for step (2) as you described while the loop for fixing up can be used for step (3). Any thoughts?

[bulbazord]

since the framework fix script already unifdefs the headers then the unifdefing that we have before we run the script should not be necessary anymore

Agreed, this is unnecessary.

I like having a staging directory to work from for the framework headers specifically

What do you like about the staging directory? With framework-header-fix, could we not just do something like this?

python3 framework-header-fix.py -i ${header_from_source_dir} -o ${BUILD_DIR}/bin/LLDB.framework/Headers <other options here>

Then you could have a dependency graph of:

lldb
^ (Depends on)
lldb-framework-headers
^ (Depends on)
{ fixup-SBAddress-header, fixup-SBBreakpoint-header, fixup-SBDebugger-header, ...}

[chelcassanova]

What do you like about the staging directory? With framework-header-fix, could we not just do something like this?

Honestly I just liked the idea of having all headers in a specific area in the build directory to use as input, but at this point it's way easier to just pull from the source dir as input and write to the framework as output here.

[bulbazord]

The dependency management here is incorrect. Nothing ties lldb-framework-fixup-header-${header_basename} to the custom command below.

For example, if you tried to build the target lldb-framework-fixup-header-SBAddress, the only thing we know is that the target depends on include/lldb/API/SBAddress.h. It doesn't automatically invoke the command to fixup that header and place it in the framework directory.

[chelcassanova]

Forgot to swap the custom command's dependency to be on the basename target instead of liblldb itself here.

[JDevlieghere]

There's no more "staging directory", right?

[chelcassanova]

Not for this loop since we only care about the basenames.

[JDevlieghere]

You can fold this into the loop below and do the transform on every iteration instead of creating a separate list?

[chelcassanova]

Good point, changing this.

[JDevlieghere]

The new code duplicates a lot of the existing logic in API/CMakeLists.txt. Wouldn't it be simpler to add the framework code to the existing loop behind if(LLDB_BUILD_FRAMEWORK)?

[chelcassanova]

The new code duplicates a lot of the existing logic in API/CMakeLists.txt. Wouldn't it be simpler to add the framework code to the existing loop behind if(LLDB_BUILD_FRAMEWORK)?

That can be done, I think we'd also have to move the top-level targets for fixing up framework headers to that file behind that if-gate as well.

[bulbazord]

The new code duplicates a lot of the existing logic in API/CMakeLists.txt. Wouldn't it be simpler to add the framework code to the existing loop behind if(LLDB_BUILD_FRAMEWORK)?

That can be done, I think we'd also have to move the top-level targets for fixing up framework headers to that file behind that if-gate as well.

Let's do that then. As-is, I have a hard time reasoning about the differences and overlaps between the logic you're modifying and the logic in source/API/CMakeLists.txt.

[bulbazord]

Shouldn't input_header just be ${header} here?

[chelcassanova]

Sorry, this is outdated now. Should be updated!