No access to Internet from VMs on minione latest / ubuntu 22.04.3

[alexxspb]

Hello!
We spent many days to resolve only one problem - no access to Internet from VMs on minione latest / ubuntu 22.04.3
It produces on various machines with 1 or 2 LAN + wifi interfaces.

Our network settings:
Host

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp46s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:e0:4c:88:3f:27 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.72/24 metric 100 brd 192.168.1.255 scope global dynamic enp46s0
       valid_lft 25199sec preferred_lft 25199sec
3: enp47s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:e0:4c:88:3f:28 brd ff:ff:ff:ff:ff:ff
4: wlp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:d8:23:16:e5:90 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.129/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp45s0
       valid_lft 24566sec preferred_lft 24566sec
    inet6 fe80::50b6:e4e9:cc5c:7f63/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
5: minionebr: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5e:f1:77:77:aa:ec brd ff:ff:ff:ff:ff:ff
    inet 172.16.100.1/24 brd 172.16.100.255 scope global minionebr
       valid_lft forever preferred_lft forever
    inet6 fe80::5cf1:77ff:fe77:aaec/64 scope link
       valid_lft forever preferred_lft forever
6: minionebr-nic: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master minionebr state UNKNOWN group default qlen 1000
    link/ether 7a:76:39:44:56:9f brd ff:ff:ff:ff:ff:ff
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:4e:88:23:c4 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
8: one-0-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master minionebr state UNKNOWN group default qlen 1000
    link/ether fe:00:ac:10:64:02 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc00:acff:fe10:6402/64 scope link
       valid_lft forever preferred_lft forever


default via 192.168.1.1 dev enp46s0 proto dhcp src 192.168.1.72 metric 100
default via 192.168.1.1 dev wlp45s0 proto dhcp metric 600
169.254.0.0/16 dev wlp45s0 scope link metric 1000
172.16.100.0/24 dev minionebr proto kernel scope link src 172.16.100.1
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.1.0/24 dev enp46s0 proto kernel scope link src 192.168.1.72 metric 100
192.168.1.0/24 dev wlp45s0 proto kernel scope link src 192.168.1.129 metric 600
192.168.1.1 dev enp46s0 proto dhcp scope link src 192.168.1.72 metric 100

VM

root@localhost:~# ping cnet.com
PING cnet.com (34.149.196.126) 56(84) bytes of data.
^C
--- cnet.com ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2029ms

root@localhost:~# dig cnet.com

; <<>> DiG 9.18.12-0ubuntu0.22.04.1-Ubuntu <<>> cnet.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4381
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;cnet.com.			IN	A

;; ANSWER SECTION:
cnet.com.		14688	IN	A	34.149.196.126

;; Query time: 0 msec
;; SERVER: 172.16.100.1#53(172.16.100.1) (UDP)
;; WHEN: Sat Oct 07 19:41:49 UTC 2023
;; MSG SIZE  rcvd: 53

root@localhost:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:ac:10:64:02 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 172.16.100.2/24 brd 172.16.100.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::acff:fe10:6402/64 scope link
       valid_lft forever preferred_lft forever
root@localhost:~# ip r
default via 172.16.100.1 dev eth0 onlink
172.16.100.0/24 dev eth0 proto kernel scope link src 172.16.100.2

[xorel]

Can you check your iptables (iptablas-save) on the OpenNebula host?

[alexxspb]

sudo iptables-save
# Generated by iptables-save v1.8.7 on Mon Oct  9 11:08:29 2023
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Completed on Mon Oct  9 11:08:29 2023
# Generated by iptables-save v1.8.7 on Mon Oct  9 11:08:29 2023
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:one-0-0-i - [0:0]
:one-0-0-o - [0:0]
:one-1-0-i - [0:0]
:one-1-0-o - [0:0]
:opennebula - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -m physdev --physdev-is-bridged -j opennebula
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A one-0-0-i -m state --state RELATED,ESTABLISHED -j RETURN
-A one-0-0-i -j RETURN
-A one-0-0-i -j DROP
-A one-0-0-o -m state --state RELATED,ESTABLISHED -j RETURN
-A one-0-0-o -j RETURN
-A one-0-0-o -j DROP
-A one-1-0-i -m state --state RELATED,ESTABLISHED -j RETURN
-A one-1-0-i -j RETURN
-A one-1-0-i -j DROP
-A one-1-0-o -m state --state RELATED,ESTABLISHED -j RETURN
-A one-1-0-o -j RETURN
-A one-1-0-o -j DROP
-A opennebula -m physdev --physdev-in one-1-0 --physdev-is-bridged -j one-1-0-o
-A opennebula -m physdev --physdev-out one-1-0 --physdev-is-bridged -j one-1-0-i
-A opennebula -m physdev --physdev-in one-0-0 --physdev-is-bridged -j one-0-0-o
-A opennebula -m physdev --physdev-out one-0-0 --physdev-is-bridged -j one-0-0-i
-A opennebula -j ACCEPT
COMMIT
# Completed on Mon Oct  9 11:08:29 2023
# Generated by iptables-save v1.8.7 on Mon Oct  9 11:08:29 2023
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.16.100.0/24 ! -d 172.16.100.0/24 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT
# Completed on Mon Oct  9 11:08:29 2023

[xorel]

Hmm, it all looks good. Except for the wlp45s0 I have a very similar setup which works just fine.
Strange DNS works from the VM but the other traffic is blocked.

[alexxspb]

Yeap, it's very strange for us, we thought that it would work out of the box at new hardware (Chatreey it12 mini-pc, i9 12900h, 2 LAN 2.5G, Wifi 6) and fresh OS installation (Ubuntu Server 22.04.1, Ubuntu Desktop 22.04.3), though we had the same problem on 2 our old PCs (i5 3570, GA-Z68AP-D3 with 1 LAN + wifi) in different places (and another routers).