SDN/Tunnel status problem

M

MichiFry

Active Member
Jun 25, 2019
3
0
41
40
Hello,

my logfiles are currently flooded with this lines:
Jan 14 12:12:41 hpve1 pvestatd[3564]: sdn status update error: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end
of string)") at /usr/share/perl5/PVE/Network/SDN/Zones.pm line 202.
Jan 14 12:12:50 hpve1 pvestatd[3564]: sdn status update error: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end
of string)") at /usr/share/perl5/PVE/Network/SDN/Zones.pm line 202.
Jan 14 12:13:01 hpve1 pvestatd[3564]: sdn status update error: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end
of string)") at /usr/share/perl5/PVE/Network/SDN/Zones.pm line 202.
Jan 14 12:13:10 hpve1 pvestatd[3564]: sdn status update error: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end
of string)") at /usr/share/perl5/PVE/Network/SDN/Zones.pm line 202.
...


This line is generated because I added an gre tunnel. The tunnel and networking is working as expected, only Proxmox has a problem reading the current state.

Code: 
# ifquery -c -a -o json
error: main exception: 'IPNetwork' object has no attribute 'config'

# ifquery -c -o json gre1
error: main exception: 'IPNetwork' object has no attribute 'config'

# ifquery -c -o json vmbr0
[
  {
    "name": "vmbr0",
    "addr_method": "static",
    "addr_family": "inet",
    "auto": true,
    "config": {
      "bridge-ports": "enp7s0",
      "bridge-fd": "0",
      "bridge-stp": "no",
      "address": "x.x.x.x/x"
    },
    "config_status": {
      "bridge-ports": "pass",
      "bridge-fd": "pass",
      "bridge-stp": "pass",
      "address": "pass"
    },
    "status": "pass"
  }
]

The problem is ifquery with --check AND --json. With only one specified it is working:
Code: 
# ifquery -o json gre1
[
    {
        "name": "gre1",
        "addr_method": "tunnel",
        "addr_family": "inet",
        "auto": true,
        "config": {
            "address": "10.10.167.202/30",
            "tunnel-mode": "gre",
            "tunnel-local": "10.50.1.11",
            "tunnel-endpoint": "10.50.0.254",
            "tunnel-ttl": "255",
            "tunnel-dev": "enp7s0.4000",
            "mtu": "1375"
        }
    }
]


# ifquery -c gre1
auto gre1
iface gre1 inet tunnel                                              [pass]
        tunnel-mode gre                                             [pass]
        tunnel-local 10.50.1.11/32                                  [pass]
        tunnel-endpoint 10.50.0.254/32                              [pass]
        tunnel-ttl 255                                              [pass]
        tunnel-dev enp7s0.4000                                      [pass]
        mtu 1375                                                    [pass]
        address 10.10.167.202/30                                    [pass]

/etc/network/interfaces
Code: 
auto gre1
iface gre1 inet tunnel
        address 10.10.167.202/30
        tunnel-mode gre
        tunnel-local 10.50.1.11
        tunnel-endpoint 10.50.0.254
        tunnel-ttl 255
        tunnel-physdev enp7s0.4000
        mtu 1375

I also tried to switch it to an ipip tunnel, but the problem persists.
Latest Proxmox version with all updates installed.

What can I do?
Thank you!
 
Hi, this seems to be an issue with ifupdown2 - can you post the interfaces file that leads to this error?
 
/etc/network/interfaces
Code: 
auto lo
iface lo inet loopback

iface enp7s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address x.x.x.x/x
        gateway x.x.x.x
        bridge-ports enp7s0
        bridge-stp off
        bridge-fd 0


auto enp7s0.4000
iface enp7s0.4000 inet static
        address 10.50.1.11/24
        mtu 1400
        up ip route add 10.50.0.0/16 via 10.50.1.1 dev enp7s0.4000
        down ip route del 10.50.0.0/16 via 10.50.1.1 dev enp7s0.4000

auto gre1
iface gre1 inet tunnel
        address 10.10.167.202/30
        tunnel-mode gre
        tunnel-local 10.50.1.11
        tunnel-endpoint 10.50.0.254
        tunnel-ttl 255
        tunnel-physdev enp7s0.4000
        mtu 1375

source-directory /etc/network/interfaces.d/

/etc/network/interfaces.d/sdn
Code: 
#version:9

auto vnet10
iface vnet10
        address 10.10.161.254/24
        post-up iptables -t nat -A POSTROUTING -s '10.10.161.0/24' -o vmbr0 -j SNAT --to-source x.x.x.x
        post-down iptables -t nat -D POSTROUTING -s '10.10.161.0/24' -o vmbr0 -j SNAT --to-source x.x.x.x
        post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        alias mgmt
        ip-forward on

auto vnet11
iface vnet11
        address 10.10.162.254/24
        post-up iptables -t nat -A POSTROUTING -s '10.10.162.0/24' -o vmbr0 -j SNAT --to-source x.x.x.x
        post-down iptables -t nat -D POSTROUTING -s '10.10.162.0/24' -o vmbr0 -j SNAT --to-source x.x.x.x
        post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        alias server
        ip-forward on


I also tried it on a new install without anything else added, same problem:
Code: 
auto lo
iface lo inet loopback

auto enp7s0
iface enp7s0 inet static
        address x.x.x.x/x
        gateway x.x.x.x

auto gre1
iface gre1 inet tunnel
        address 10.10.167.202/30
        tunnel-mode gre
        tunnel-local 10.50.1.11
        tunnel-endpoint 10.50.0.254
        tunnel-ttl 255
        mtu 1375

Code: 
# ifquery -c -o json gre1
error: main exception: 'IPNetwork' object has no attribute 'config'


Also tried it without the "tunnel-" (only mode, local endpoint, ttl) and different combinations "tunnel-physdev", .. nothing helped.
But the gre tunnel always works :)
 
Yes, the issue is most likely in the ifquery tool itself, where the GRE status gets queried. Could you create a bug report on this? I'll try to look into it.
 
Hi,

just wanted to say that I got the same issue.

For testing purposes I set up an ipip interface (also handled by tunnel.py module of ifupdown2) and the same error happens.

I did some debugging (although I lack knowledge of Python). The class ifaceJsonEncoderWithStatus() in ifupdown/iface.py shows up twice. The first time the object 'o' seems ok, o.config points to the config section of the json (see below output of pdb). In the second call - which I think shouldn't happen at all - 'o' just contains the value of 'tunnel-local' and is of type IPNetwork.

Code: 
(Pdb) run gre1 -c -o json
Restarting /usr/share/ifupdown2/__main__.py with arguments:
        gre1 -c -o json
> /usr/share/ifupdown2/__main__.py(26)<module>()
-> import os
(Pdb) c
> /usr/share/ifupdown2/ifupdown/iface.py(310)default()
-> retifacedict = OrderedDict([])
(Pdb) p o.config
OrderedDict([('up', ['ip route add 10.222.111.0/24 dev gre1']), ('tunnel-mode', ['gre']), ('tunnel-local', [10.99.88.18/32]), ('tunnel-endpoint', [10.99.88.2/32]), ('tunnel-dev', ['eno1.4004']), ('mtu', ['1374']), ('address', ['10.11.11.18/27'])])
(Pdb) c
> /usr/share/ifupdown2/ifupdown/iface.py(310)default()
-> retifacedict = OrderedDict([])
(Pdb) p o.config
*** AttributeError: 'IPNetwork' object has no attribute 'config'
(Pdb) p o
10.99.88.18/32
(Pdb)

My GRE interface definition:

Code: 
auto gre1
iface gre1 inet tunnel
    address  10.11.11.18/27
    tunnel-mode     gre
    tunnel-endpoint 10.99.88.2
    tunnel-local    10.99.88.18
    tunnel-dev eno1.4004
    # hetzner vswitch doc -> mtu 1400 for VLAN interface
    # set a bit under, can't remember the calculation
    mtu      1374
 
Ok, I was able to track the issue down. Keep in mind than I'm not a programmer and my Python knowledge is very, very limited.

In ifupdown/iface.py:325
retconfig[k] = v[0] if len(v) == 1 else v
converted the values to string:
retconfig[k] = str(v[0] if len(v) == 1 else v)

That's it. Without it fails at the first IP address/network object, I assume since the value is not a valid Python type in case of the tunnel-endpoint and tunnel-local, it's not happy.

Without converting to string, the OrderedDict looked like the following:

Code: 
retifacedict.config OrderedDict([
    ('name', 'gre1'),
    ('addr_method', 'tunnel'),
    ('addr_family', 'inet'),
    ('auto', True),
    ('config',
        {'up': 'ip route add 10.222.111.0/24 dev gre1',
        'tunnel-local': 10.99.88.18/32,
        'tunnel-mode': 'gre',
        'tunnel-dev': 'eno1.4004',
        'tunnel-endpoint': 10.99.88.2/32,
        'mtu': '1374',
        'address': '10.11.11.18/27'}
    ),
    ('config_status',
        {'up': '',
        'tunnel-local': 'pass',
        'tunnel-mode': 'pass',
        'tunnel-dev': 'pass',
        'tunnel-endpoint': 'pass',
        'mtu': 'pass',
        'address': 'pass'}
    ),
    ('status', 'pass')
])

Maybe (hopefully) there is a more elegant solution? But the errors are gone. Maybe modify the IPNetwork object to return a string for the = operation?

Code: 
root@pve-fluffy /usr/share/ifupdown2 # ifquery gre1 -c -o json
[
  {
    "name": "gre1",
    "addr_method": "tunnel",
    "addr_family": "inet",
    "auto": true,
    "config": {
      "up": "ip route add 10.222.111.0/24 dev gre1",
      "tunnel-local": "10.99.88.18/32",
      "tunnel-mode": "gre",
      "tunnel-dev": "eno1.4004",
      "tunnel-endpoint": "10.99.88.2/32",
      "mtu": "1374",
      "address": "10.11.11.18/27"
    },
    "config_status": {
      "up": "",
      "tunnel-local": "pass",
      "tunnel-mode": "pass",
      "tunnel-dev": "pass",
      "tunnel-endpoint": "pass",
      "mtu": "pass",
      "address": "pass"
    },
    "status": "pass"
  }
]
 
You must log in or register to reply here.
Top Bottom