-
Notifications
You must be signed in to change notification settings - Fork 7.1k
Initial port of the SuperNOVAS C/C++ library (v1.5.0-rc8) #47819
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
7eea519
to
f2ec923
Compare
@microsoft-github-policy-service agree |
bfb8d5b
to
165c32f
Compare
ports/supernovas/portfile.cmake
Outdated
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include") | ||
|
||
if(VCPKG_LIBRARY_LINKAGE STREQUAL "static") | ||
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/bin" "${CURRENT_PACKAGES_DIR}/debug/bin") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is removed here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @dg0yt,
Good question. I copied this from the existing calceph port, which is a dependency. This was in relation to build errors, like:
/home/pumukli/git/vcpkg/ports/supernovas/portfile.cmake: warning: ${CURRENT_PACKAGES_DIR}/debug/include should not exist. To suppress this message, add set(VCPKG_POLICY_ALLOW_DEBUG_INCLUDE enabled)
note: If this directory was created by a build system that does not allow installing headers in debug to be disabled, delete the duplicate directory with file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include")
/home/pumukli/git/vcpkg/ports/supernovas/portfile.cmake: warning: ${CURRENT_PACKAGES_DIR}/debug/share should not exist. Please reorganize any important files, then delete any remaining by adding `file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share")`. To suppress this message, add set(VCPKG_POLICY_ALLOW_DEBUG_SHARE enabled)
So it's doing what the build errors suggests to do, and they do make the errors disappear...
Anyway, any further insight is much appreciated.
I'm also still having a hard time with the multi-platform builds. (The upstream package builds fine with CMake on Linux, FreeBSD, MacOSX, and Windows -- but the vcpkg does not, especially with the calceph
dependency added).
-
When I enable the
calceph
dependency, many platform builds fails trying to buildcalceph
. Which seems strange, given thecalceph
is a fully integrated port that should build for all platforms exceptuwp
based on its vcpkg config... -
Without the
calceph
dependency, I get a build failure on Android, which seems to be a C standard issue. I think I can fix that upstream in the source, but it may require some trial and error to find just what workaround is needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PS. Yes, the android build seems to fail due to a known omission in some versions of the android clib. I'll work around that for the next rc (next week probably)...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Patching the upstream and disabling the calceph dependency now results in a clean CI (below)...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
existing calceph port, which is a dependency
It is not declared as a dependency. Am I missing something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not a dependency at the moment. But it's a dependency of a feature that is currently disabled, at least until calceph is healthy again. I'll re-enable the feature and the dependency as soon as calceph allows it.
P.S. Thanks for the review! ;-)
b4bdd70
to
4869ba9
Compare
Thanks @dg0yt. I went ahead and made (I'm a little confused about default features still. The docs explain how it is used, but not how you define a feature to be 'default'. My guess would be that it's by setting such options in |
This looks at least plausible to me: supernovas provides CMake targets:
# this is heuristically generated, and may not be correct
find_package(supernovas CONFIG REQUIRED)
target_link_libraries(main PRIVATE supernovas::supernovas)
supernovas provides pkg-config modules:
# SuperNOVAS astrometry library
supernovas |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is kind of a lot of product code to be touching in a patch. Since you appear to be upstream would you be willing to merge those changes upstream rather than trying to patch them after the fact here?
The problem is that we can't speak to the correctness of, or otherwise maintain, patches with so much product code change.
But for the big patches I would merge this.
-#include <semaphore.h> | ||
|
||
/// \cond PRIVATE | ||
+#if defined(_PTHREAD_) || defined(__unix__) || defined(__unix) || defined(__APPLE__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couple of suggestions on this part of the patch. (This is not 'Billy the vcpkg maintainer' speaking, this is 'Billy the random C++ programmer speaking', if that makes sense)
mtx_Xxx
are problematic names to use because those names are in the C standard. This may be resolved if you flipped the order and put the__STDC_VERSION__ >= 201112L
branch first.- It's very strange to declare an object-like macro in some branches but function-like macros in other branches. It's clear that this infrastructure only works for 1 argument given the
#else
branch, so making them all function-like macros seems doable. - You may wish to consider adding something like the following, as long as you don't require your mutex to be a recursive mutex. (I kept your names despite the first comment)
#elif defined(WIN32)
#include <windows.h>
# define mtx_init (x) InitializeSRWLock(x)
# define mtx_lock (x) AcquireSRWLockExclusive(x)
# define mtx_unlock (x) ReleaseSRWLockExclusive(x)
# define THREAD_SAFE 1
typedef SRWLOCK mtx_t;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @BillyONeal,
Yes, trying to make mutexes portable is a pain. The C11 threads.h
was meant to improve on it, but then clang does not support it, and while mtx_lock()
returns int
on Linux but void
on Windows (apparently). So it's a mess no matter...
As for the first comment, you are right. I'll change the names in the next rc upstream.
The current patches were a proof of concept, really. I'll have these done more properly in the next upstream release candidate (in ~2 days). I'll incorporate your suggestion for the WIN32
version, and also re-think on how to make the 'non-thread-safe' branch better without turning the code into a mess of #if
/ #else
/ #endif
blocks...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mtx_lock()
returnsint
on Linux butvoid
on Windows (apparently).
A function-like macro doesn't care about the return type :)
As for the first comment, the order should be safe so long as threads.h is not included in the source, other than the appropriate feature branch. (I.e., even though mtx_* are C11 standard, they should only be defined if threads.h is included).
I vaguely recall a general rule saying to not macroize standard library names but I'm not finding it now so 🤷
The current patches were a proof of concept, really.
Sounds good! I see now you said this in the PR description already and I missed it. (Given all our baseline issues last week I'm being paranoid and trying to look at all the things again...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right that the macro doesn't care for the return type, but the original upstream code checked the return values of the mutex lock call, which then failed for the windows builds... It really is walking a tight rope with the mutexes...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I'll have one more dress rehearsal with rc8 in ~2 days. Hopefully it will go smoothly. Even so, let's not merge that yet either, but wait for the final 1.5.0 release (around Oct 29). Once ready, I change the PR status to 'Ready for review' and tag you in a note also...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nice-to-haves.
ports/supernovas/cmake.patch
Outdated
include(GNUInstallDirs) | ||
include(CMakePackageConfigHelpers) | ||
include(FeatureSummary) | ||
+include(FindThreads) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Either
+include(FindThreads) |
or
+include(FindThreads) | |
+find_package(Threads) |
find_package
sets interface variables for find modules.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be just my confusion, but I thought find_package(Threads)
needs the FindThreads
CMake module (https://cmake.org/cmake/help/latest/module/FindThreads.html). Hence, I thought you need both (as the patch had), but I might be wrong. Perhaps the module inclusion is not needed...
Edit: It seems like it was my misunderstanding of the CMake documentation. FindThreads
is just a label for a find_package
use case. It's not a CMake module...
ports/supernovas/cmake.patch
Outdated
) | ||
|
||
+ if(CMAKE_USE_PTHREADS_INIT) | ||
+ target_compile_definitions(${PLUGIN} PRIVATE _PTHREAD_) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would generally suggest a unique prefix instead of _
.
+ target_compile_definitions(${PLUGIN} PRIVATE _PTHREAD_) | |
+ target_compile_definitions(${PLUGIN} PRIVATE SUPERNOVAS_USE_PTHREAD) |
and similar in the source code. Avoids clashes with third-party public macros, and facilitates grep
etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. Will change in upstream, for the next rc....
ports/supernovas/cmake.patch
Outdated
supernovas | ||
${EPHEM_LIB} | ||
${LIBM} | ||
+ Threads::Threads |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reminder, didn't check:
Using targets is best practice. But this needs to be complemented with find_dependency(<Pkg> ...)
(without REQUIRED
) in the installed CMake config, at least for static library linkage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess you mean find_package(Threads)
without the REQUIRED
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I really mean find_dependency
which is available after include(CMakeFindDependencyMacro)
.
https://cmake.org/cmake/help/latest/module/CMakeFindDependencyMacro.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I'm a little confused here (just to show how ignorant I am). The CMake docs on FindThreads
has the following example:
set(THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package(Threads)
target_link_libraries(example PRIVATE Threads::Threads)
Do you mean that I need to add a find_dependency(Threads::Threads)
after the find_package()
before using it in the target_link_libraries()
? If so, I wish the CMake documentation were more accurate on this...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Edit: After some googling, it seems like the conventional way is to add find_dependency(Threads)
to cmake/supernovasConfig.cmake.in
. Or, do you want it in portfile.cmake
also?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In CMakeLists.txt, use find_package
. And REQUIRED
as needed.
The portfile has vcpkg_cmake_config_fixup
, so I assume supernovas exports and installs a CMake config package.
Now when you link to the provided Threads::Threads
, it will be referenced in the interface link libraries of export targets (at least for static linkage). (You can inspect the installed files.)
Now downstream users do find_package(supernovas CONFIG REQUIRED)
.
They do not know that supernovas uses this target. But supernovas does, and it is the CMake config's responsibility to provide this target. It could do find_package(Threads)
. But getting REQUIRED
, QUIET
, and diagnostics play well with the capabilities of find_package
is a lot of work, and find_dependency
is the recommended wrapper doing the job.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
portfile doesn't need it. It is just a script in CMake language, not CMake project mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was really helpful thanks. I'm pretty new to CMake, having started with it just about a month ago. So the clarification is really welcome ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added find_dependency(Threads)
upstream for the solsys-calceph
(and solsys-cspice
) components. Would you like to check if that looks OK?
https://github.com/Smithsonian/SuperNOVAS/blob/main/cmake/supernovasConfig.cmake.in
Threads is needed only for these optional plugin components. (The solsys-cspice
component isn't packaged yet, but I just saw that there is a cspice
vcpkg already, so I'll try add that next also).
ports/supernovas/portfile.cmake
Outdated
REF "v${VERSION}" | ||
SHA512 4babecc41fbdfa515f2aea2f07f6b052605d16c1c3d6aa3ab44d13de6cf4ccd1a59f4e72e269623fd4ea73b83b72ff7bd6068c8cb09866a535412e50ee040446 | ||
HEAD_REF main | ||
PATCHES android.patch solsys-calceph.patch cmake.patch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typical port style:
PATCHES android.patch solsys-calceph.patch cmake.patch | |
PATCHES | |
android.patch | |
solsys-calceph.patch | |
cmake.patch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The patches will disappear soon, in the next rc...
ports/supernovas/portfile.cmake
Outdated
vcpkg_cmake_configure( | ||
SOURCE_PATH "${SOURCE_PATH}" | ||
OPTIONS | ||
-DBUILD_TESTING=OFF ${FEATURE_OPTIONS} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typical port style:
-DBUILD_TESTING=OFF ${FEATURE_OPTIONS} | |
-DBUILD_TESTING=OFF | |
${FEATURE_OPTIONS} |
ports/supernovas/portfile.cmake
Outdated
if(VCPKG_LIBRARY_LINKAGE STREQUAL "static") | ||
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/bin" "${CURRENT_PACKAGES_DIR}/debug/bin") | ||
endif() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is removed here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not 100% sure, except that the static build spat out errors when this was not added.
Perhaps, it is related to the docedit
executable created during the build, which just makes a version of the Github README.md
that is a more standard standalone markdown. It's just a build-time tool, to remove badges and get rid of github style decorations. It should not be installed (but perhaps it did get installed by CMake).
I'll double check what files CMake installs for static builds, and see if there is something I can do to remedy it upstream...
vcpkg_cmake_config_fixup(CONFIG_PATH "lib/cmake/${PORT}") | ||
vcpkg_fixup_pkgconfig() | ||
|
||
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping this together.
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include") | |
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include") | |
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share") |
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share") | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share") | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vcpkg definitely gave errors if without it, and explicitly suggested adding this line...
f8b4471
to
f1963cf
Compare
The additional |
Hi @BillyONeal and @dg0yt, Thank you both for all the work you put into helping this package take shape. The next (and hopefully last) upstream release candidate is now here. All patches are gone, and I have incorporated all your suggestions upstream. The plan is to get this PR ready to merge next week with the final 1.5.0 release. A few outstanding items, questions:
|
@dg0yt, Here I try to build against the cspice package (the original before your PR), and I get errors like (see the CI below):
Given that cspice seems to install the headers, I'm not sure what's amiss. Maybe you can help me find the reason for the error... |
The line before is the command line:
It doesn't contain The vcpkg CMake toolchain adds search paths to inform CMake. But it doesn't have magical power to know the right paths and path lists for individual build system targets. This is left to the project's build system. Supernovas' CMakeLists.txt (or a find module) needs to add something like: find_path(CSPICE_INCLUDE_DIR "cspice/SpiceUsr.h" REQUIRED)
target_include_directories(core PRIVATE "${CSPICE_INCLUDE_DIR}") And similar for other dependencies. (The |
Thanks for the suggestion. Some progress on the cspice feature inclusion, but the windows dll builds against the cspice lib still fail. It's not clear to me from the logs why (it simply states FAILED with an error code that means nothing to me):
Do you have any insights? |
@dg0yt, A separate question. Regarding the file removals in The files removed are effectively the For the vcpkg debug builds I can only really set build config variables for Or, perhaps, sticking with the removals in |
Similar issue, but now in the link step:
The lib is not given with full path, but there is no effort to provide a search path to the linker. CMakeLists.txt has find_library(cspice cspice REGISTRY_VIEW TARGET REQUIRED) which sets a variable named
carrying the literal But what you actually want is to use the result of
(Do you really need this plugin function? It might adds complexity when tracing prolems. Perhaps YAGNI.) |
By default, vcpkg build two configurations of each port, debug and release. There are release-only triplets which skip the debug build. And Development != Debug. |
I guess not. I just thought it might be nice to add the feature given that there is a |
I was refering to the custom cmake function handling plugins in the build system, not to the cspice feature/plugin. I often encounter such convenience functions when tracing problems, and it can be a PITA to find out how they are meant work and how to make them work. Just another layer of indirection. |
I see. Without the function I'd have repeated blocks of the same code -- notwidthstanding the library name (cspice / calceph). I see your point about tracing errors, but I really despise repeated code blocks, since if one wants to change some part of it, there is high chance that one either forgets to update all occurrences, or else that the changes won't end up being exactly the same among the occurrences. It's a common source of stupid errors, which the use of a common function eliminates entirely... Anyway, I've made changes upstream again to use |
Hi,
I would like to add the SuperNOVAS C/C++ library to the vcpkg registry. I am the creator / maintainer of SuperNOVAS, and this is my first ever vcpkg PR. (As such mea culpa if I did not get everything right, and I look forward to any guidance you might provide).
This initial PR is for the latest and likely last release candidate for the upcoming v1.5.0 release, expected around November 1 (or perhaps a bit earlier). Thus, this PR is also a final dress-rehearsal to ensure that the upstream is fully ready for vcpkg listing. If there are any issues with the upstream regarding the vcpkg inclusion, please let me know so it may be fixed for the final release. Otherwise, please merge this PR as appropriate, and I will update the package again as soon as the final v1.5.0 is released.
Thanks,
-- Attila
find_package
calls are REQUIRED, are satisfied byvcpkg.json
's declared dependencies, or disabled with CMAKE_DISABLE_FIND_PACKAGE_Xxx.vcpkg.json
matches what upstream says.vcpkg.json
matches what upstream says../vcpkg x-add-version --all
and committing the result.