VIP VoIP - Status Page

Issues for inbound calls are still ongoing.

We are still awaiting further updates from Amazon Web services with these issues via our carrier partners Magrathea where engineer teams are continuing to liaise around a solution

We do not expect any further updates until the morning now but will continue to monitor things and feedback any updates as we receive them

VIP VoIP support team

Further update from our Carrier Partner - Magrathea
Following on from the issues that were experienced on Friday 2nd May, please find RFO details below.

We sincerely apologise for the inconvenience that this disruption will have caused you – thank you for your support and patience during this period.

Please do not reply to this email, if you have any queries please contact us on support@vipvoip.co.uk

Post Incident Review

Incident Symptom: Inability to process inbound/outbound SIP calls

Date of Incident: Friday 2nd May 2025

Incident Duration: 14:00 to 19:45 (intermittent and varying impact)

Incident Timeline:

Magrathea customers experienced issues with inbound SIP calls on Friday from approximately 14:15.

Numbers that had failover routing in place continued to function and the fault was intermittent meaning the severity and extent was unclear until approximately 14:44 at which point we issued our first service notice.

Service was initially restored a few minutes later with an all clear notice sent at 14:54. However the issue reappeared and impacted inbound SIP calls again intermittently up to 16:21 when it was cleared.

At approximately 17:50 we received reports of number translation targets being out of sync and a few minutes later reports of outbound calls failing. Both of these issues impacted a smaller number of clients but the same inbound call failures reappeared intermittently alongside. We finally reported the issue fully resolved at 19:48 and continued monitoring before issuing an ‘all clear’ notice at 21:00.

Incident Summary:

The primary cause of this incident was a memory leak condition in the systems responsible for monitoring and reporting errors. This had the effect of depriving operational memory to systems responsible for core network routing decisions causing them to malfunction.

Because the fault occurred in the monitoring systems themselves it frustrated our engineers attempt to quickly resolve the problem and in addition as the fault was intermittent in nature not all of our bank of four independent systems were affected.

These systems have run unchanged for many years without issue and so the issue was completely unexpected. We have taken immediate action to ensure that it cannot reoccur as well as planning additional separate ways to detect this type of issue promptly if it occurs elsewhere in the network.

Separately to this a minor fault occurred in our network during a similar time period which caused changes requested through our numbering API to not propagate through the network correctly. This fault has been isolated to a new network component that was undergoing our final test stages prior to deployment. This was not resolved with our normal speed due to our focus on the more significant fault above.

This incident has been resolved.

Please see the latest update from our carrier partners Magrathea regarding this below:

***************************************

Dear all

The issue with inbound calls appears to be cleared.  A full RFO will be sent as soon as we have concluded our investigations.

We are sincerely sorry for the impact this has caused you and your customers this afternoon.

***************************************

We apologise for any inconvenience caused.

If you have any queries please contact us on support@vipvoip.co.uk

We have been made aware that the previous issue raised regarding inbound calls failing is NOT yet resolved.

Magrathea did recently send us a notice to say they were but have since retracted this.

These current issues do seem to be intermittent and will not be affecting all numbers but we are aware of reports that some users are still experiencing issues affecting calls for inbound routes that are routed via Magrathea

Our team are in contact with Magrathea who have engineers investigating this as a priority and we will provide a further update shortly.

We apologise for the inconvenience

Please do not reply to this email, if you have any queries please contact us

on support@vipvoip.co.uk

Kind regards

VIP VoIP Support Team

W: www.vipvoip.co.uk