HOME       INTEGRATION ECOSYSTEM       AUTHENTICATORS       RESOURCES
Classic - Radius over VPN Service Incident
Incident Report for SafeNet Trusted Access
Postmortem
Posted Apr 24, 2023 - 00:19 UTC

Resolved
We have completed the necessary work to fully restore the RADIUS over VPN IPsec service in the STA legacy UK Datacenter after identifying a series of network device failures. The restoration required both physical and logical reconfiguration of the network connectivity.

However, we recommend that you continue to use the legacy Canadian Datacenter as your primary VPN endpoint.

We plan to begin decommissioning our legacy datacenters in the UK (primary) and Canada (secondary) towards the end of June 2023. These datacenters currently host our VPN IPsec tunnels. We want to inform you that we have finalized planning for the decommissioning process, and the last date of support from these legacy datacenters will be June 30, 2023.

What action item is required from your end?

If you are currently in the process of migrating or creating tickets for migration, please work with our STA Customer Support team for guidance on the next steps. Our team will connect you with the technical contact from STA Operations who can assist you with the migration process.

If you need additional support reach out via https://supportportal.thalesgroup.com/csm or call: US: 800-545-6608 | International: +1-410-931-7520
Posted Apr 18, 2023 - 17:35 UTC
Update
We strongly advise that customers continue using Canada as the primary DC for their Radius over IPsec VPN authentications until we have implemented a permanent fix on the legacy UK DC site. The service, although accessible on the UK side, is not currently reliable.

We will be performing physical maintenance at the UK DC site in the coming hours. We will provide updates as they become available.
Posted Apr 18, 2023 - 11:34 UTC
Update
We strongly advise that customers continue using Canada as the primary DC for their Radius over IPsec VPN authentications until we have implemented a permanent fix on the legacy UK DC site. The service, although accessible on the UK side, is not currently reliable.

We will be performing physical maintenance at the UK DC site in the coming hours. We will provide updates as they become available.
Posted Apr 18, 2023 - 05:10 UTC
Update
Customers connecting back to legacy UK DC should be advised that we are operating in a workaround state. As such, we highly recommend you stay on the legacy Canada DC, if possible until we have fully rectified the issue. We are continuing to test, monitor and troubleshoot.
Posted Apr 18, 2023 - 04:24 UTC
Update
We are seeing positive results after completing the workaround. Radius over VPN traffic is being successfully processed. We will test and monitor to ensure continued stability.
Posted Apr 18, 2023 - 04:01 UTC
Update
It will take some time to complete the work required to restore service. We will provide updates when we have completed the work.
Posted Apr 18, 2023 - 03:22 UTC
Update
We are continuing to work on a fix for this issue.
Posted Apr 18, 2023 - 03:21 UTC
Update
We have implemented a workaround and are currently testing.
Posted Apr 18, 2023 - 03:16 UTC
Update
We are continuing to troubleshoot and work towards resolution. We will provide updates as they become available.
Posted Apr 18, 2023 - 03:06 UTC
Update
The attempt to implement the workaround was unsuccessful. We are exploring other avenues while troubleshooting. We will provide updates as they become available.
Posted Apr 18, 2023 - 02:01 UTC
Update
We are working on a workaround to the current issue. We do not yet have an estimated time to repair but will provide one once available.
Posted Apr 18, 2023 - 00:41 UTC
Update
We are continuing to troubleshoot and implement a workaround.
Posted Apr 17, 2023 - 23:43 UTC
Update
We are currently working in the UK DC to restore the Radius over VPN service being served from the legacy UK DC

As a reminder, customers can still access the Radius over VPN service from the legacy Canada DC endpoint.
Posted Apr 17, 2023 - 23:15 UTC
Update
We are still working towards resolution. We will provide updates as they become available.
Posted Apr 17, 2023 - 22:53 UTC
Identified
The issue appears to be a network layer issue and we are attempting restorative efforts.
Posted Apr 17, 2023 - 21:31 UTC
Update
We are continuing to troubleshoot and work towards resolution. We will provide updates as they become available.
Posted Apr 17, 2023 - 20:29 UTC
Investigating
The SafeNet Trusted Access monitoring system has detected a problem with "Radius over VPN Service " for customers solely accessing the legacy UK DC endpoint through IPSec VPN tunnels.

Customers who have the legacy Canada DC endpoint configured will not be impacted.

The Thales incident response team is already investigating this problem and we will update you as soon as we have updates.
Posted Apr 17, 2023 - 19:40 UTC
This incident affected: Classic Service Zone (Authentication Services).