Hi, first of all many thanks to your great programs.
We have been running a single pritunl server (now v1.30.3116.68) on ubuntu 20.04.3 machine, where dozens of clients can connect, with multiple device feature enabled.
The pritunl server suddenly denied all multiple-device connection of any user, including newly-created one, making an error "unable to assign ip address, pool full". When there is no connection from a user (so the web UI says offline) it accepted one connection but not more.
This is the snippet of the log, which denies the second connection of a user:
[winter-thunder-9503][2022-04-20 14:23:27,699][ERROR] Unable to assign ip address, pool full
server_id = "6225cbdbbfbdafd4c3915****"
instance_id = "6231a364b965eac1ab09****"
user_id = "62341bf5b965eac1ab0a***"
multi_device = true
replica_count = 1
network = "10.8.0.0/24"
user_count = 22
There seems to be no mongodb issue (from the mongod.service and mongodb log I can tell).
After I re-started the pritunl server via web UI, the symptom was gone and started to work well.
What can be the source of the problem and how should I avoid this?
This issue should be fixed in a recent v1.30 update. If you had a previous version when the issue occurred it should be fixed after restarting the server. Updating the package requires running sudo systemctl restart pritunl to load the latest version.
We’re running on 1.30.3292.22-0ubuntu1~focal and we get the unable to assign ip address, pool full error a few hours after we disable the Allow Multiple Devices option on the server advanced config screen. If we re-enable Allow Multiple Devices, no problems for anyone to connect.
How many devices are connecting to the server? Each user that is attached to the server is assigned a static IP even when not connected. This will reduce the available IP addresses for additional devices.
We had issues with “Unable to assign ip address, pool full” when a user tried to connect with multiple devices after even after extending the network range of a server.
I checked the database and noticed that pool_cursor was set to the start of the network range, even though there were a lot of unused IP addresses in the network.
According to the source code it is never reset, unless you restart the server.
In order to avoid a restart of the server, I fixed this by running:
All users that are in an organization attached to the server are assigned a static IP address starting from the beginning of the subnet. When multiple devices are enabled additional user devices are assigned a temporary IP address starting from the end of the subnet. There was an issue with this in versions before v1.30.3331.78 that had left behind an expire index in the clients_pool database collection. The design was changed to clear the user ID from the IP document to make it available for reuse, this index was removing those documents. To fix this all the hosts must be updated then all the servers must be stopped and started.
We (the same instance as in @pascal-hofmann post) experienced the issue again. It maxed out at the /22 networks broadcast address while only having a few dozens users connected.
It seems the pool_cursor is never reset / cleaned up during runtime. Did it manually again.