No scm info shown after adding project

[pa5446530a]

Describe the bug
Repositories added to an existing webapp do not show scm info. The repositories are indexed properly and .git folder is present.
Both opengrok_tools.utils.opengrok.add_project and opengrok-projadm -a have the same behavior. The new repositories are indexed using opengrok-reindex-project.

To Reproduce
Steps to reproduce the behavior:

• Clone new repository
• Add repository to existing webapp
• Index the new project

Expected behavior
Scm info is shown for the new repostitory.

Screenshots

New repositories are shown under Project, ones present upon initial indexing are shown properly.

Additional context
Running in Docker, version 1.7.30

[vladak]

Retrieve fresh configuration from the web app http://localhost:8080/source/api/v1/configuration and see if there are any RepositoryInfo objects matching the project. If not, the repositories were not discovered for some reason. Running the indexer in verbose mode might reveal more.

[pa5446530a]

Thanks for the pointers. The projects in question lack a RepositoryInfo entry in the configuration.
Increasing the indexer verbosity did yield a possible cause: INFO: Done scanning for repositories, found 999 repositories (took 5.89 seconds)
I know for sure there are more than 999 repositories, is there a cap on this number? If that's the case, the new repositories likely simply did not make the cut.

[vladak]

There is definitely no cap. Is there something special about these undiscovered repos you can deduce by running basic git commands underneath the respective directories ? (I assume these should be Git repos)

[vladak]

For the record, this is how a directory is considered to be a Git repository:

opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/history/GitRepository.java

Lines 402 to 409 in b33822c

	boolean isRepositoryFor(File file, CommandTimeoutType cmdType) {
	if (file.isDirectory()) {
	File f = new File(file, Constants.DOT_GIT);
	// No check for directory or file as submodules contain '.git' file.
	return f.exists();
	}
	return false;
	}

So basically as long as it contains a .git entry (which can be either file or directory), it is considered to be a Git repository.

[vladak]

The only limit I can think of right now is the scanning depth limit - if the repo question is too deep down the directory tree, it might not be discovered with default settings.

See scanningDepth in https://github.com/oracle/opengrok/wiki/Indexer-configuration#indexer-tunables

[pa5446530a]

All repositories are in /opengrok/src so depth is not an issue. Git commands work as expected in both older and newly cloned repositories.
I suspect the issue lies somewhere in the per-project indexing process since git info is processed correctly when reindexing the entire webapp. Doing things this way is not feasible however since part of the indexed content is quite dynamic.
These are the commands used when adding new content:
opengrok-projadm -b /opengrok -U http://localhost:8080/source -a project
opengrok-reindex-project --printoutput --jar /opengrok/lib/opengrok.jar -U http://localhost:8080/source -P project -- -r dirbased -G -m 64 --leadingWildCards on -S -c /usr/local/bin/ctags -i '*.pdb' -i '*.zip' -U http://localhost:8080/source project

[vladak]

All repositories are in /opengrok/src so depth is not an issue.

Do you mean the repositories are actually directly underneath source root ? E.g. /opengrok/src/foo and such ? (so that /opengrok/src/foo/.git actually exists)

How does the output from the opengrok-projadm command look like ? Normally it would report the repositories of newly added project.

Are there any interesting messages in the web app logs ?

[pa5446530a]

Yes, all of the problematic projects have a structure like /opengrok/src/project/.git. Each repository contains one or two folders with content that needs to be indexed.
opengrok-projadm outputs only Adding project * and then Refreshing configuration. Attempting to read repositories via the webapp api returns an empty list.
The webapp logs do not contain anything unusual, INFO [http-nio-8080-exec-9] org.opengrok.web.api.v1.controller.ProjectsController.addProject adding project * is the only line that shows up after running opengrok-projadm -a.

I attempted to replicate this behavior in a fresh container and everything works fine. I am investigating the differences in configuration, will update if something relevant turns up but it might be that the issue lies somewhere in our config.