Auth timestamp expired - client fails to reconnect

anekos · November 2, 2022, 3:43pm

We have been getting issues on a monthly base where some of our (linux) clients fail to reconnect / sync the profile due to temporary network failure (according to the logs). Our monitoring system didn’t report any network related issues for that period.

The clients are installed with pritunl-client (not electron)
pritunl-client: Installed: 1.3.3300.95-0ubuntu1~jammy

The connection works again when we (re)start the profile with pritunl-client start {profile}
Is there a way to configure infinite connection retries for the client?

[2022-08-29 11:14:54][INFO] ▶ profile: Profile exit, reconnecting ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-08-29 11:14:54][INFO] ▶ profile: Disconnecting ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-08-29 11:14:55][INFO] ▶ profile: Disconnected ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-09-21 13:10:25][INFO] ▶ main: Service starting ◆ version="1.2.3236.80"
[2022-09-21 13:10:28][INFO] ▶ profile: Connecting ◆ dynamic_firewall=false ◆ mode="ovpn" ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-09-21 13:10:28][ERRO] ▶ profile: Failed to sync system profile ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
sprofile: Sync profile connection error
Get "https://xxx.xxx.xxx.xxx/key/sync/6287960c0efea7bf09ad8097/6287960d0efea7bf09ad80b3/622a1492027d7399eebee5f1/b81dfbced5a8d9c6b588767cc98c5519": dial tcp xxx.xxx.xxx.xxx:443: connect: network is unreachable
ORIGINAL STACK TRACE:
github.com/pritunl/pritunl-client-electron/service/sprofile.(*Sprofile).syncProfile
	/go/src/github.com/pritunl/pritunl-client-electron/service/sprofile/sprofile.go:436 +0x8ee4ec
github.com/pritunl/pritunl-client-electron/service/sprofile.(*Sprofile).Sync
	/go/src/github.com/pritunl/pritunl-client-electron/service/sprofile/sprofile.go:491 +0x902bf9
github.com/pritunl/pritunl-client-electron/service/profile.(*Profile).Start
	/go/src/github.com/pritunl/pritunl-client-electron/service/profile/profile.go:1244 +0x902a00
github.com/pritunl/pritunl-client-electron/service/profile.SyncSystemProfiles.func1
	/go/src/github.com/pritunl/pritunl-client-electron/service/profile/utils.go:391 +0x91454c
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x465ee0
[2022-09-21 13:10:38][INFO] ▶ profile: Connecting ◆ dynamic_firewall=false ◆ mode="ovpn" ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-21 13:10:47][INFO] ▶ profile: Disconnecting ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-09-21 13:10:48][INFO] ▶ profile: Disconnected ◆ profile_id="rc3vyuccpkidn3lpr22lu8b4prep2wmm"
[2022-09-21 13:34:08][INFO] ▶ profile: Profile exit, reconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-21 13:34:08][INFO] ▶ profile: Disconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-21 13:34:09][INFO] ▶ profile: Disconnected ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-21 13:34:12][INFO] ▶ main: Service starting ◆ version="1.3.3290.45"
[2022-09-21 13:34:15][INFO] ▶ profile: Connecting ◆ dynamic_firewall=false ◆ mode="ovpn" ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 15:03:28][INFO] ▶ profile: Profile exit, reconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 15:03:28][INFO] ▶ profile: Disconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 15:03:28][INFO] ▶ profile: Disconnected ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 17:03:31][INFO] ▶ main: Service starting ◆ version="1.3.3300.95"
[2022-09-26 17:03:34][INFO] ▶ profile: Connecting ◆ dynamic_firewall=false ◆ mode="ovpn" ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 22:02:55][INFO] ▶ profile: Profile exit, reconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 22:02:55][INFO] ▶ profile: Disconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 22:02:55][INFO] ▶ profile: Disconnected ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 22:03:03][INFO] ▶ profile: Connecting ◆ dynamic_firewall=false ◆ mode="ovpn" ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-09-26 22:03:06][ERRO] ▶ profile: Failed to sync system profile ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
sprofile: Sync profile connection error
Get "https://52.174.60.222/key/sync/61f93cb4caee1b0e01524c42/6226406503c32cce1705b6e7/61f944c1caee1b0e0152594b/a17d8a00c6b3ec08a36ae2cd1c7caa72": dial tcp 52.174.60.222:443: connect: no route to host
ORIGINAL STACK TRACE:
github.com/pritunl/pritunl-client-electron/service/sprofile.(*Sprofile).syncProfile
	/go/src/github.com/pritunl/pritunl-client-electron/service/sprofile/sprofile.go:441 +0x8f07cc
github.com/pritunl/pritunl-client-electron/service/sprofile.(*Sprofile).Sync
	/go/src/github.com/pritunl/pritunl-client-electron/service/sprofile/sprofile.go:496 +0x90431e
github.com/pritunl/pritunl-client-electron/service/profile.(*Profile).Start
	/go/src/github.com/pritunl/pritunl-client-electron/service/profile/profile.go:1159 +0x904100
github.com/pritunl/pritunl-client-electron/service/profile.SyncSystemProfiles.func1
	/go/src/github.com/pritunl/pritunl-client-electron/service/profile/utils.go:386 +0x9162ac
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x465ee0
[2022-10-26 22:06:09][ERRO] ▶ profile: Stopping system profile due to authentication errors ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-10-26 22:06:09][INFO] ▶ profile: Disconnecting ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"
[2022-10-26 22:06:09][INFO] ▶ profile: Disconnected ◆ profile_id="a1qwx8m828morsmylahkjx46ltj5qqny"

zach · November 2, 2022, 9:38pm

This should only occur if the profile receives authentication errors which typically don’t resolve on retries. Does this profile have authentication configured?

anekos · November 3, 2022, 8:17am

Hi Zach,

Do you mean 2fa authentication?
These clients are set to bypass secondary authentication.
We use pritunl on our production (headless) linux servers, so 2fa authentication is not an option.

The issues mostly appear every 30 days when the client tries to re-authenticate / reconnect?

On the profile audit we see the following error message:
User connection to "prod-vpn" denied. Auth timestamp expired

From the audit log below only the last connection (Nov 2 16:18) was done manually, the rest were done by the system.
Both the clients and server are set to UTC.

Edit:
I just saw that a similar question is asked here:

Our server has the default value for the setting, which is 12 hours:

$ sudo pritunl get app.auth_time_window
app.auth_time_window = 43200

Regards,
Anestis

zach · November 3, 2022, 4:45pm

I need more information about the configuration of the server.

Is either Pritunl Authentication Cache or OpenVPN Authentication Cache enabled in the top right settings?

In the advanced settings of the user having the issue what is the Type set to?
Is either the PIN or YubiKey ID set on the user?

anekos · November 3, 2022, 7:47pm

Below are the user settings. All servers have the same settings.
Type is local
No pin or YubikeyID is used.

Below are the general settings.
We use Azure SSO, but this is only for normal (human) users’ accounts.
Openvpn and pritunl authentication cache is disabled.

zach · November 3, 2022, 9:18pm

I have found the cause of this issue, it will be fixed in the next release.

anekos · November 4, 2022, 7:59am

Hi Zach,

Thanks so much for looking into this issue.

The fix will be on the next client release?
Can you advise when this is planned for?

Anestis

zach · November 7, 2022, 12:39pm

This release is now available.

anekos · November 7, 2022, 2:45pm

Hi Zach,

Is the fix in client version 1.3.3343.50?
Or is it on the server?

We are using the pritunl-client (no gui) version. I can see a new version on pritunl-client-electron only, the pritunl-client package doesn’t seem to have been updated with the latest release.

pritunl-client-electron: got version 1.3.3343.50 on github, but apt-cache reports 1.3.3329.81-0ubuntu1~jammy as latest.
pritunl-client has not been updated since July…

Can you advise?

Thanks,
Anestis

zach · November 7, 2022, 8:47pm

The Linux builds are now available on the repository.