Sonarr remains inactive, after fix isn't reachable through reverse proxy

The problem

I restarted sonarr from the sonarr web interface. It went down but never came back up. After waiting for a while, I reinstalled it (keeping configuration settings), still remains inactive. Then I reinstalled without keeping configuration settings, still inactive.

When reinstalling the logs are:

┬ ┬┬ ┬┬ ┬ ┬ ┬┌┬┐┌─┐┌─┐┌┐ ┌─┐┌─┐┬┌┐┌┌┐┌┌─┐┬─┐ ┌─┐┌─┐┌┬┐
 │││││││││ ├─┤ │ ├─┘│  ├┴┐├┤ │ ┬│││││││├┤ ├┬┘ │  │ ││││
 └┴┘└┴┘└┴┘o┴ ┴ ┴ ┴  └─┘└─┘└─┘└─┘┴┘└┘┘└┘└─┘┴└─o└─┘└─┘┴ ┴
                __  ___
  /\ |_ _ |\/|./     | _  _ ||_/.|_
 /--\|_(_)|  ||\__   |(_)(_)|| \||_

AtoMiC Sonarr Installer Script

Press [Enter] key to continue or Ctrl+C to exit...


--->Stopping Sonarr...
Nothing found to stop

--->Removing Sonarr Systemd Startup scripts...
sonarr.service SystemD script not found

--->Removing Sonarr Init.d startup scripts...
/etc/init.d/sonarr init script not found

--->Removing Sonarr default config scripts...
/etc/default/sonarr default script not found

--->Adding Sonarr repository...
Sonarr repository already exists

AtoMiC Mono Installer Script

--->Adding Mono repository...
Mono repository already exists

--->Refreshing packages list...
Done

--->Installing prerequisites for Mono...
Reading package lists... Done
Building dependency tree
Reading state information... Done
libmono-cil-dev is already the newest version (5.12.0.226-0xamarin3+ubuntu1604b1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

--->Installing Mono...
Reading package lists... Done
Building dependency tree
Reading state information... Done
mono-complete is already the newest version (5.12.0.226-0xamarin3+ubuntu1604b1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
---> Mono installation complete.

--->Installing Sonarr...
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nzbdrone

--->Stopping Sonarr...
Nothing found to stop

--->Autostart configuring...
Using systemd

--->Removing Sonarr Init.d startup scripts...
/etc/init.d/sonarr init script not found

--->Removing Sonarr default config scripts...
/etc/default/sonarr default script not found

--->Enabling Sonarr Autostart at Boot...
Created symlink from /etc/systemd/system/multi-user.target.wants/sonarr.service to /etc/systemd/system/sonarr.service.
sonarr.service SystemD script added and enabled

--->Starting Sonarr...
sonarr.service is now: inactive

systemctl status sonarr.service yields:

● sonarr.service - Sonarr
   Loaded: loaded (/etc/systemd/system/sonarr.service; enabled; vendor preset: enabled)
   Active: inactive (dead) (Result: exit-code) since Sat 2018-06-16 12:16:47 CEST; 5min ago
  Process: 9040 ExecStart=/usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser (code=exited, status=2)
 Main PID: 9040 (code=exited, status=2)

Jun 16 12:16:47 media systemd[1]: sonarr.service: Unit entered failed state.
Jun 16 12:16:47 media systemd[1]: sonarr.service: Failed with result 'exit-code'.
Jun 16 12:16:47 media systemd[1]: sonarr.service: Service hold-off time over, scheduling restart.
Jun 16 12:16:47 media systemd[1]: Stopped Sonarr.
Jun 16 12:16:47 media systemd[1]: sonarr.service: Start request repeated too quickly.
Jun 16 12:16:47 media systemd[1]: Failed to start Sonarr.

When trying to run the start command for sonarr manually (/usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser) the error is:

Cannot open assembly '/opt/NzbDrone/NzbDrone.exe': No such file or directory.

Checking /opt with ls -l /opt:

total 24
drwxrwsr-x 53 ismay ismay 4096 Jun 15 17:39 atomic-toolkit
drwxrwsr-x  5 ismay ismay 4096 Jan 19 12:09 nzbget
drwxrwsr-x 12 ismay ismay 4096 Jun 16 09:59 nzbhydra
drwxrwsr-x  6 ismay ismay 4096 Jan 14 13:10 nzbtomedia
drwxrwsr-x  4 ismay ismay 4096 Jun 15 17:30 radarr
drwxrwsr-x  4 ismay ismay 4096 Feb 23 13:36 radarr_03-03-2018_13-35

So it's not there. Attempting a manual install with sudo apt-get install nzbdrone:

Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nzbdrone

Manually installed and restarted it with:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys FDA5DFFC
sudo echo "deb http://apt.sonarr.tv/ master main" | sudo tee /etc/apt/sources.list.d/sonarr.list
sudo apt-get update
sudo apt-get install nzbdrone 

On reboot it now starts as expected, but still isn't accessible from another computer. Running access details from atk yields:

 ┬ ┬┬ ┬┬ ┬ ┬ ┬┌┬┐┌─┐┌─┐┌┐ ┌─┐┌─┐┬┌┐┌┌┐┌┌─┐┬─┐ ┌─┐┌─┐┌┬┐
 │││││││││ ├─┤ │ ├─┘│  ├┴┐├┤ │ ┬│││││││├┤ ├┬┘ │  │ ││││
 └┴┘└┴┘└┴┘o┴ ┴ ┴ ┴  └─┘└─┘└─┘└─┘┴┘└┘┘└┘└─┘┴└─o└─┘└─┘┴ ┴
                __  ___
  /\ |_ _ |\/|./     | _  _ ||_/.|_
 /--\|_(_)|  ||\__   |(_)(_)|| \||_

AtoMiC Sonarr Access Details

Press [Enter] key to continue or Ctrl+C to exit...

--->Searching for possible port numbers...
Default port: 8989 found in AtoMiC-ToolKit config.
Error: near line 1: database disk image is malformed

--->Finding Sonarr System details...

You may be able to access Sonarr on any of the following URLs...
--->http://[..............]:8989 from anywhere (requires port forwarding on router)
--->http://192.168.1.5:8989 from your local network
--->http://media:8989 from your local network
--->0.0.0.0/8989 on this system
Actual port numbers could be different. Check your settings file: /home/ismay/.config/NzbDrone/config.xml
If SSL is enabled, then use HTTPS instead of HTTP in the above URLs.

Your Sonarr credentials are...
--->Username: Incompatibility. Cannot determine username.
--->Password: Incompatibility. Cannot determine password.

Checking Sonarr status...
SYSTEMD sonarr.service found.
sonarr.service is currently: active
Sonarr is configured to be accessible outside localhost.

Thank you for using the AtoMiC ToolKit.
Please visit https://gitter.im/htpcBeginner/AtoMiC-ToolKit to discuss or raise any issues.

Press [Enter] key to continue or Ctrl+C to exit...

Any idea why other local devices cannot connect to the service? Other services on the same device, like radarr and nzbget can be reached without problems.

My Linux distribution and version is:

Ubuntu Server 16.04.4 LTS

I'm seeing this behaviour on:

• Was working now broken

I'm not a dummy, so I've checked these!

• I'm using the latest version of AtoMiC-ToolKit
• All my Linux distro packages are up to date and I don't see any errors while updating.
• I've checked there are no current open issues relating to my problem!

I tried tunneling over ssh to the media server ssh me@media.home -L 8989:127.0.0.1:8989 and then accessing it via localhost:8989, that yields:

channel 3: open failed: connect failed: Connection refused

So whilst sonarr is running, the webserver is refusing to connect on port 8989. So does sonarr spawn its own webserver? Or is atk responsible for that?

Ok, so the sonarr database was corrupted. After restoring to another backup it works as expected.

So while that's not got anything to do with atk, other parts in the process (such as having to manually install nzbdrone) should probably be caught by atk. So my particular issue was resolved, but hopefully atk could catch some of these things automatically in the future. Hope this helps!