Tips on authentication limiter

cristian · August 25, 2023, 2:07pm

Hi, I installed a pritunl enterprise panel on a single dedicated server where I created 13 different servers to which about 400 users will connect (for each server) for a total of about 5,400 users. All servers are connected to a single Hosts.
Sometimes, for various reasons, I happen to restart all the servers and it happens that for each single server only the first 20 users connect quickly and it does not allow (immediately) that others connect which happens then but very slowly. I read this article about this problem

but honestly I didn’t understand if it’s enough to send these commands as they are indicated or should they be modified according to my needs?

I would be grateful to anyone who provides me with this precious information, if possible with an example based on my aforementioned needs, thank you.

zach · August 25, 2023, 8:59pm

This may not be an authentication limit issue, you would see authentication limit error messages in the top right logs in the web console.

The authentication limits are per user, if required the commands can be run to increase the limits. The numbers shown on that page are the defaults. The ttl shouldn’t be increased, this would result in longer lockouts. The limiter can safely be increased to 100. The ttl is a database expire index, that would require restarting at least one host to update the database index. The limiter count should take effect on all hosts in 1-2 minutes after a settings sync. These commands only need to be run on one host in a cluster and the options are then saved in the database.

sudo pritunl set app.auth_limiter_ttl 600
sudo pritunl set app.auth_limiter_count_max 100

cristian · August 25, 2023, 9:33pm

thanks and what do you think could cause this problem?

zach · August 25, 2023, 9:48pm

Check the logs in the top right of the web console for the error message.

cristian · August 25, 2023, 9:55pm

If it helps you today I did this test: Sent only the “sudo pritunl clear-auth-limit” command I stopped and restarted a server and I noticed that no user could connect and the error appeared in the server log “AUTH FAILED” at which point I sent the other 2 commands “sudo pritunl set app.auth_limiter_ttl 600” and “sudo pritunl set app.auth_limiter_count_max 20” And so immediately all the users started to connect. What happens?

cristian · August 25, 2023, 10:10pm

Find this error in the log:

21:1:39,556][ERROR] Exception on /event [GET]
Traceback (most recent call last):
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 2528, in wsgi_app
response = self.full_dispatch_request()
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1823, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/auth/app.py”, line 10, in _wrapped
return call(*args, **kwargs)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/handlers/event.py”, line 28, in event_get
return utils.jsonify(event.get_events(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/event.py”, line 25, in get_events
for event in messenger.subscribe(‘events’, cursor_id=cursor,
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/helpers.py”, line 51, in _wrapped
for value in call(*args, **kwargs):
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/messenger.py”, line 141, in subscribe
for doc in cursor:
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1280, in next
if len(self.__data) or self._refresh():
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1195, in _refresh
self.__send_message(q)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1078, in __send_message
response = client._run_operation(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1515, in _run_operation
return self._retryable_read(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1617, in _retryable_read
return func(session, server, sock_info, secondary_ok)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1511, in _cmd
return server.run_operation(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/server.py”, line 133, in run_operation
_check_command_response(first, sock_info.max_wire_version)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/helpers.py”, line 180, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Executor error during find command :: caused by :: CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(2445395), full error: {‘ok’: 0.0, ‘errmsg’: ‘Executor error during find command :: caused by :: CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(2445395)’, ‘code’: 136, ‘codeName’: ‘CappedPositionLost’}
[summer-plateau-7505][2023-08-25 21:57:39,556][ERROR] Exception on /event [GET]
Traceback (most recent call last):
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 2528, in wsgi_app
response = self.full_dispatch_request()
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1823, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/flask/app.py”, line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/auth/app.py”, line 10, in _wrapped
return call(*args, **kwargs)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/handlers/event.py”, line 28, in event_get
return utils.jsonify(event.get_events(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/event.py”, line 25, in get_events
for event in messenger.subscribe(‘events’, cursor_id=cursor,
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/helpers.py”, line 51, in _wrapped
for value in call(*args, **kwargs):
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pritunl/messenger.py”, line 141, in subscribe
for doc in cursor:
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1280, in next
if len(self.__data) or self._refresh():
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1195, in _refresh
self.__send_message(q)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/cursor.py”, line 1078, in __send_message
response = client._run_operation(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1515, in _run_operation
return self._retryable_read(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1617, in _retryable_read
return func(session, server, sock_info, secondary_ok)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/mongo_client.py”, line 1511, in _cmd
return server.run_operation(
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/server.py”, line 133, in run_operation
_check_command_response(first, sock_info.max_wire_version)
File “/usr/lib/pritunl/usr/lib/python3.9/site-packages/pymongo/helpers.py”, line 180, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Executor error during find command :: caused by :: CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(2445395), full error: {‘ok’: 0.0, ‘errmsg’: ‘Executor error during find command :: caused by :: CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(2445395)’, ‘code’: 136, ‘codeName’: ‘CappedPositionLost’}

zach · August 28, 2023, 11:54pm

Those errors should automatically correct.

cristian · August 29, 2023, 3:08am

Ok and for error…"AUTH FAILED” ?

zach · August 29, 2023, 9:35pm

There is no auth failed error, more information would be included in either the server output or system logs in the top right. You need to send the complete information for the error. The errors in the previous logs are unrelated issues likely from a browser tab left open attempting to reuse an expired event cursor which will correct on it’s own with a retry.

cristian · August 30, 2023, 6:33am

Thanks zach, in fact the errors that I have pointed out to you refer to the “limits” speech while the authentication error actually appears to me in the server log section with authentication errors that I was unable to capture.