Incident History

May 2024

Friday 24th May 2024

UKAT SIMs Connectivity 17052024 UKAT Incident Notification - RESOLVED

Description: This is to inform you that the incident regarding UKAT activations is now resolved.

AT&T engineers identified a misconfiguration on our private network which caused IP allocations to fail upon provisioning.

If you continue to face any issues with AT&T activations, please raise a ticket so we can assist you.

Our sincerest apologies for any inconvenience this may have caused.

Services affected: Provisioning

Location: Global

Start Time: 24-05-2024 17:00 UTC

Friday 17th May 2024

UKAT SIMs Connectivity 17052024 UKAT Incident Notification - NEW

Description: We're currently experiencing some issues with some activations on the UKAT network.

Investigations are currently ongoing and all relevant teams have been involved

The vendor has been involved in the troubleshooting process and they are also investigating it with priority.

We will keep you updated on the news.

Services affected: Provisioning

Location: Global

Start Time: 17-05-2024 18:00 UTC

Wednesday 15th May 2024

ROTN SIMs Connectivity 14052024 Disturbance with Globe Telecom in Philippines [RESOLVED]

Description: The issue has been solved. Tests of the network are now successful, and traffic patterns have improved.

Services affected: Data

Location: Philippines

End Time: 2024-05-14 22:39 UTC

Tuesday 14th May 2024

ROPD SIMs Connectivity 14052024 ROPD Incident Notification [RESOLVED]

Description: This is to inform you that the issue with ROPD/RODB/ROPDT/ROTEU data connectivity service is now resolved

The issue was signalling congestion, during which a small proportion of subscribers may have experienced issues re-establishing connectivity or registering to the network.

There will be an emergency maintenance tonight to implement additional network capacity and help prevent future issues.

Services affected: Data

Location: Global

End Time: 2024-05-14 17:00 (UTC)

ROPD SIMs Connectivity 14052024 ROPD Incident Notification [NEW]

Description:IoT Service Team has identified some performance issues on the ROPD/RODB/ROPDT/ROTEU data connectivity service using the data641003 APN. ​

We're seeing some intermittent packet loss on one of our Packet Gateways, which has been already escalated to our vendor. ​

Services remain stable, but the issue is affecting devices re-registering or establishing data sessions.

Investigations and recovery measures are currently ongoing and the Incident has been prioritized on our side.

Services affected: Data

Location: Global

Start Time: 14-05-2024 14:00

Monday 13th May 2024

ROTN SIMs Connectivity ROTN Data Disturbance Notification

Description:

On May 13th, an undersea cable has been damaged which causes disruptions in East Africa on subscriptions roaming in Vodacom's network.

Services affected:

Data, Voice, SMS

Location:

This issue might be seen in, but not limited to: Kenya, Tanzania, Rwanda, Uganda, Malawi and Madagascar as well.

Start Time:

2024-05-13 08:21 (UTC)

Thursday 2nd May 2024

AirOn360 IoT Suite 02052024 UKJ Incident Notification-iotsuite

Description: 02052024 IoTSuite Planned Maintenance Notifications NEW Services Affected: API Location: GLOBAL Start time: 2024-05-02T13:00:00+02:00 End time: 2024-05-02T15:53:00+02:00 Duration: 0d 2h 53m

April 2024

Thursday 25th April 2024

UKJ SIMs Connectivity 25042024 UKJ API Degradation - RESOLVED

Description: This is to notify you that the API degradation issue on our UKJ network is now resolved

All placed orders have been executed and processing should now be normal.

Connectivity and data transmissions has NOT been impacted for any productive devices.

Start Time: 2024-04-25 15:30 UTC

UKJ SIMs Connectivity 25042024 UKJ API Degradation - NEW

This is to notify you about an unplanned API degradation on our UKJ network. You may experience an issue when trying to Activate, Suspend or Purge your UKJ SIMs at the moment. The situation has been escalated to the network engineers. We will share with you further updates as soon as these become available.

Start date: 2024-04-25 15:30 UTC

Monday 22nd April 2024

ROTEU SIMs Connectivity ​22042024 PGW1 Performance Issues - RESOLVED

Dear Customer,

This is to notify you that the issue has been resolved

Start Date: 22/04/2024 15:00hs UTC

End Date: 25/04/2024 09:00hs UTC

ROPDB SIMs Connectivity ​22042024 PGW1 Performance Issues - NEW

Dear Customer,

This is to notify you that yesterday at 15:00hs UTC we detected some issues on our PGW1. This lead to an unplanned Emergency Maintenance affecting APN data641003 for under 10 minutes causing no connectivity.

This was escalated to the vendor and has been resolved. Connections are stable and coming back to normal. The engineers are still monitoring it as some intermittent disconnections have happened. RFO is yet to be shared.

Start Date: 22/04/2024 15:00hs UTC

End Date: TBD

ROPDT SIMs Connectivity ​22042024 PGW1 Performance Issues - NEW

Dear Customer,

This is to notify you that yesterday at 15:00hs UTC we detected some issues on our PGW1. This lead to an unplanned Emergency Maintenance affecting APN data641003 for under 10 minutes causing no connectivity.

This was escalated to the vendor and has been resolved. Connections are stable and coming back to normal. The engineers are still monitoring it as some intermittent disconnections have happened. RFO is yet to be shared.

Start Date: 22/04/2024 15:00hs UTC

End Date: TBD

ROPD SIMs Connectivity ​22042024 PGW1 Performance Issues - NEW

Dear Customer,

This is to notify you that yesterday at 15:00hs UTC we detected some issues on our PGW1. This lead to an unplanned Emergency Maintenance affecting APN data641003 for under 10 minutes causing no connectivity.

This was escalated to the vendor and has been resolved. Connections are stable and coming back to normal. The engineers are still monitoring it as some intermittent disconnections have happened. RFO is yet to be shared.

Start Date: 22/04/2024 15:00hs UTC

End Date: TBD

Sunday 14th April 2024

UKAT SIMs Connectivity 14042024 UKAT Incident Notification - NEW

Description: We have identified an issue with new activations for UKAT SIMs. The activations are failing with different errors, such as 'Error subscribing asset'. At the moment, the issue appears to be when attempting to change a SIM from Inactive to Active Status. SIMs that are already in Active status or that were suspended and reactivated do not seem to be impacted.

Services affected: Data

Location: Global

Start Time: 2024-04-14 13:00 UTC

Tuesday 2nd April 2024

UKJ SIMs Connectivity 02042024 UKJ API Degradation - RESOLVED

Description: This is to notify you the issue of the API degradation has been resolved. Please let us know if you have any questions on this. Sorry for the issue and thank you for understanding.

Services affected: Provisioning

Location: Global

Start Time: 2024-04-02 04:00 UTC

March 2024

Monday 25th March 2024

ROTEU SIMs Connectivity 25032024 Emergency Maintenance - RESOLVED

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: 2024-03-23 21:20 UTC

ROPDT SIMs Connectivity 25032024 Emergency Maintenance - Resolved

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: 2024-03-23 21:20 UTC

ROPD SIMs Connectivity 25032024 Emergency Maintenance - RESOLVED

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: 2024-03-23 21:20 UTC

ROPD SIMs Connectivity GeneralincidentTemplate

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: {{ duration }}

ROPDT SIMs Connectivity GeneralincidentTemplate

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: 2024-03-25 21:00 UTC

ROTEU SIMs Connectivity 25032024 Emergency maintenance - NEW

Description: We have identified an issue causing transmission errors on the network interface on one of our Core Routers at the LD1 Datacenter. This datacenter has traffic from the carriers ROPD, ROPDB, ROTEU using our IP subnet 45.154.16.0/24.

We will be migrating traffic to an alternative path, and during 21:00 - 21:10 UTC data will be disrupted for a few minutes whilst the routing is updated.

Services affected: Data

Location: Global

Start Time: 2024-03-25 21:00 UTC

Saturday 23rd March 2024

UKAT SIMs Connectivity 22032024 AT&T Incident Notification - RESOLVED

Description: The incident regarding AT&T experiencing disturbance with data traffic on the West Coast has been resolved. Packet loss is no longer observed.

Services affected: Data

Location: US

Start Time: 2024-03-23 02:46 UTC

Friday 22nd March 2024

UKAT SIMs Connectivity 22032024 AT&T Incident Notification - NEW

Description: AT&T has acknowledged that they are experiencing disturbance with data traffic while roaming on their network at different locations on the West Coast. Customers might experience a significant degree of packet loss. Updates will follow when more information becomes available.

Services affected: Data

Location: US

Start Time: 2024-03-22 17:00 UTC

Friday 15th March 2024

UKJ SIMs Connectivity 15032024 UKJ Incident Notification (Africa) - NEW

Description: This is to notify you that several Network Operators in Africa area are experiencing service outages due to multiple submarine optic fibre cables that come into the area. Some of the cables are down while others are delivering limited capacity. This is impacting internet and data. As a result, both inbound and outbound roaming is greatly affected.

Services affected: Data

Location: Africa

Start Time: 15-03-2024 14:00 UTC

UKJ SIMs Connectivity 15032024 UKJ Incident Notification - RESOLVED

Description: All traffic on interconnect links has returned to normal levels, and services were fully restored. Our engineers will be closely monitoring for the next 24 hours.

Services affected: Data

Location: Global

Resolution Time: 15-03-2024 05:00 UTC

UKJ SIMs Connectivity 15032024 UKJ Incident Notification - UPDATE

Description: The UKJ engineers have reported back that they are in recovery mode and are seeing an improvement. We have tested and we also see much better performance. We are still monitoring however and will keep you updated until confirmed resolution

Services affected: Data

Location: Global

Update Time: 15-03-2024 04:00 UTC

UKJ SIMs Connectivity 15032024 UKJ Incident Notification - UPDATE

Description: UKJ has reported back to us that location updates are failing due to congestion. We are following-up with them frequently to find out what the engineers are doing to resolve this incident by requesting an action plan and next steps.

Services affected: Data

Location: Global

Update Time: 15-03-2024 03:00 UTC

UKJ SIMs Connectivity 15032024 UKJ Incident Notification - NEW

Description: This is to notify you that we are experiencing a degradation related to our UKJ service that is impacting Data Services. Our engineers continue to troubleshoot the issue and we will provide updates as more information becomes available.

Services affected: Data

Location: Global

Start Time: 15-03-2024 01:00 UTC

Tuesday 12th March 2024

ROPD SIMs Connectivity 12022023 ROPD Incident Notification RESOLVED

Description: The previous issue with ROPD SIM connectivity was resolved. We can see the devices back online. Please test yours and if you have any problem, make sure to raise a ticket so we can investigate. We will provide a RCA as soon as it is available.

Services affected: ROPD Connectivity

Location: Global

Start Time: 12-03-2024 15:00 UTC

ROPD SIMs Connectivity 12022023 ROPD Incident Notification NEW

Description: We would like to inform you that we noticed some issues in SIMs connectivity for a small number of ROPD SIMs. We are currently investigating the reports of data connectivity issues affecting them. Updates will be shared as soon as there's any.

If you are facing a connectivity issue with your ROPD card, please make sure to open a ticket so we can investigate.

Services affected: ROPD Connectivity

Location: Global

Start Time: 12-03-2024 13:40 UTC

Wednesday 6th March 2024

UKAT SIMs Connectivity 05032024 UKAT Provisioning Issues

Description: We would like to inform you that the issue we were facing with UKAT SIMs activation was resolved. If you are still facing any similar case, please contact our Support team via ticketing system.

Services affected: Provisioning

Location: Global

End Time: 2024-03-06 01:00 UTC

Tuesday 5th March 2024

UKAT SIMs Connectivity 05032024 UKAT Provisioning Issues

Description: We would like to inform you that we are facing some issues with the UKAT SIMs provisioning. You might experience errors in activating or suspending these SIMs. Our engineers are already looking into it. We will share updates as soon as there's any available.

Services affected: Provisioning

Location: Global

Start Time: 2024-03-05 13:00 UTC

February 2024

Friday 23rd February 2024

AirOn360 IoT Suite AT&T Major Outage - RESOLVED

Description: AT&T has restored all the services in their network at this point. We are hereby closing this notification.

Services affected: Data

Location: US

Start Time: 2024-02-22 23:30 UTC

Thursday 22nd February 2024

AirOn360 IoT Suite AT&T Major Outage

Description: AT&T has confirmed that they are having a nation wide issue with their 4G network. Their network team is working on resolving the issue. No ETR is available at this moment. We will send out another notification when more information becomes available.

Services affected: Data

Location: USA

Start Time: 2024-02-22 12:30 UTC

Thursday 15th February 2024

UKJ SIMs Connectivity 15022024 UKJ API Degradation - RESOLVED

Description: This is to notify you that this issue has been resolved. Our network provider informed us that a backlog in the provisioning flow had resulted in a delay in the orders going through. The provisioning time should now be back to normal.

Services affected: Provisioning

Location: Global

End Time: 15/02/2024 20:00 UTC

UKJ SIMs Connectivity 15022024 UKJ API Degradation - NEW

Description: This is to notify you about an API degradation on our UKJ network. You may experience an issue when trying to Activate, Suspend or Purge your UKJ SIMs at the moment. There's a delay on the order processing, so it is taking longer for the actions to be performed. The situation has been escalated to the network engineers, we will share with you further updates as soon as these become available. At the moment, we suggest you to try repeating the same action after 10minutes. Connectivity and data transmission is not affected.

Services affected: UKJ Provisioning

Location: Global

Start Time: 15/02/2024 11:00 UTC

January 2024

Friday 26th January 2024

NATM SIMs Connectivity 26012024 NATM Incident Notification - RESOLVED

Description: Data connection in Guatamala/Venezuela/Mexico

Services affected: Data/Voice/SMS

Location: Guatamala/Venezuela/Mexico

Start Time: 26 01 2024 15:00hs UTC End Time: 26-01-2024 18:00 UTC

Thursday 25th January 2024

NATM SIMs Connectivity 23012024 NATM Incident Notification - RESOLVED

Description: This is to notify you that this issue we had with NATM connectivity has been resolved. We are working with NATM for a RCA and will share it as soon as it becomes available.

Services affected: Data

Location: Mexico, Venezuela and Guatemala

Start Time: 23-01-2024 18:00 UTC End Time: 25-01-2024 04:00 UTC

NATM SIMs Connectivity 23012024 NATM Incident Notification - UPDATE 3

Description: We continue to work with the NATM team to resolve the previously reported issue. As of now, we don't have an ETA from the Network provider. We have found that SIMs from Canada seem to be impacted as well. If you are still facing data connectivity issues in your NATM SIM, please make sure you have a ticket opened with the support team.

Services affected: Data

Location: Mexico, Guatemala, Venezuela and Canada

Start Time: 23-01-2024 09:00 UTC

Wednesday 24th January 2024

NATM SIMs Connectivity 23012024 NATM Incident Notification - UPDATE 2

Description: We continue working with NATM team in order to resolve the previosuly reported issue. As of now, we don't have an ETA from NATM team. If you are still facing data disconnectivity in your NATM SIM, please make sure you have a ticket opened with the support team.

Services affected: Data

Location: Mexico, Guatemala and Venezuela

Start Time: 23-01-2024 18:00 UTC

NATM SIMs Connectivity 23012024 NATM Incident Notification - UPDATE

Description: We continue working with NATM team in order to resolve the previosuly reported issue. If you are still facing data disconnectivity in your NATM SIM, please make sure you have a ticket opened with the support team.

Services affected: Data

Location: Mexico, Guatemala, Venezuela

Start Time: 23-01-2024 18:00 UTC

Tuesday 23rd January 2024

NATM SIMs Connectivity 23012024 NATM Incident Notification - NEW

Description: We have identified partial outages on the NATM network for Mexico, Venezuela, and Guatemala.

The issue has already been escalated to our network provider and they are working to have it resolved.

We will provide more updates as soon as they become available.

Services affected: Data

Location: Mexico, Venezuela and Guatemala

Start Time: 23-01-2024 18:00 UTC

Thursday 18th January 2024

UKJ SIMs Connectivity 18012024 UKJ API Degradation - RESOLVED

Description: This is to notify you that this issue has been resolved. Our network provider informed us that a backlog in the provisioning flow had resulted in a delay in the orders going through. The provisioning time should now be back to normal.

Services affected: API, Provisioning

Location: Global

Duration: 2024-01-18 17:30 UTC

UKJ SIMs Connectivity 18012024 UKJ API Degradation - NEW

Description: This is to notify you about an API degradation on our UKJ network. You may experience an issue when trying to Activate, Suspend or Purge your UKJ SIMs at the moment. There's a delay on the order processing, so it is taking longer for the actions to be performed. The situation has been escalated to the network engineers, we will share with you further updates as soon as these become available. At the moment, we suggest you to try repeating the same action after 10minutes.

Services affected: API, Provisioning

Location: Global

Start Time: 18-01-2024 14:30 UTC

Thursday 11th January 2024

ROTN SIMs Connectivity 11012024 ROTN Incident Notification - RESOLVED - Bolivia

Description: We have been notified by the ROTN team that they have been experiencing issues with 2G and 3G for Entel Bolivia on and off since this morning. The engineers are already investigating it with the Operator.

We will provide more updates as soon as they become available.

Services affected: Data

Location: Bolivia

Start Time: 2024-01-11 16:35 UTC

Duration: 2024-01-11 22:04 UTC

UKJ SIMs Connectivity 11012024 UKJ API Degradation - RESOLVED

Description: This is to notify you that the UKJ API degradation has been resolved and we are now seeing successful API transactions on Pod IoT Suite. If you continue to experience API failures, please open a ticket via the ePortal.

Services affected: API, Provisioning

Scope: Global

Resolution time: 2024-01-11 13:00 UTC

UKJ SIMs Connectivity 11012024 UKJ API Degradation - NEW

Description: This is to notify you about an unplanned API degradation on our UKJ network. You may experience an issue when trying to Activate, Suspend or Purge your UKJ SIMs at the moment. There's a delay on the order processing, so it is taking longer for the actions to be performed.

The situation has been escalated to the network engineers, we will share with you further updates as soon as these become available.

Services affected: API, Provisioning

Scope: Global

Start time: 2024-01-11 10:00 UTC

Wednesday 10th January 2024

AirOn360 IoT Suite 10012024 - Issues with accessing the IoT Suite Portal - UK ONLY - RESOLVED

Description: We would like to inform you that the issue with the IoT Portal in the UK was resolved by our DNS partner. We have requested them the RCA and will share as soon as it becomes available.

Services affected: IoT Suite Platform

Location: UK

Start Time: 2024-01-10 16:30 UTC

AirOn360 IoT Suite 10012024 - Issues with accessing the IoT Suite Portal - UK ONLY

Description: This is to notify you about a IoT Suite Portal degradation. Customers in the UK are not being able to log into the platform. This issue is intermittent and it is affecting only the UK. The engineers are addressing the issue with top priority, and updates will be provided as soon as available.

Services affected: IoT Suite Platform

Location: UK

Start Time: 2024-01-10 15:30 UTC

December 2023

Friday 29th December 2023

AirOn360 IoT Suite 27122023 IoT Suite Portal Degradation - RESOLVED

We would like to inform you that the issue with the IoT Portal was resolved by our DNS partner. We have requested them the RCA and will share as soon as it becomes available.

The old IoT Portal URLs are still available to be used: podiotsuite.com or api.podiotsuite.com (for API users).

AirOn360 IoT Suite 27122023 IoT Suite Portal Degradation - UPDATE

We would like to inform you that we are still working on a permanent solution for the IOT Suite platform degradation. The issue was isolated and we continue working on the fix with our supplier.

As a reminder, the old URLs are available to be used in order to access the portal: podiotsuite.com or api.podiotsuite.com (for API users).

We will update you as soon as there are news.

Thursday 28th December 2023

AirOn360 IoT Suite 27122023 IoT Suite Portal Degradation - UPDATE

We would like to inform you that we are still working on a permanent solution for the IOT Suite platform degradation. The issue was isolated and we continue working on the fix with our supplier.

As a reminder, the old URLs are available to be used in order to access the portal: podiotsuite.com or api.podiotsuite.com (for API users).

We will update you as soon as there are news.

AirOn360 IoT Suite 27122023 IoT Suite Portal Degradation

We would like to inform you that our engineers are still working on a permanent solution for the IOT Suite platform degradation.

In the meantime, you can use the old URLs in order to access the portal as a work around: podiotsuite.com or api.podiotsuite.com (for API users).

As soon as the issue is fixed we will provide a new update.

AirOn360 IoT Suite Pod IOT Suite Portal Degradation - UPDATE

We would like to inform you that our Engineers are still working on a permanent solution regarding the Pod IOT Suite platform degradation. In the meantime, you can use the old URLs in order to access the portal as a work around: podiotsuite.com or api.podiotsuite.com (for API users).

Thank you for your patience and we apologise again for any inconvenience caused.

Wednesday 27th December 2023

AirOn360 IoT Suite Pod IoT Suite Portal Degradation - UPDATE

Engineers are still working on a permanent solution for the Pod IOT Suite platform degradation. In the meantime, you can use the old url in order to access the portal as a work around: podiotsuite.com or api.podiotsuite.com

As soon as the issue is fixed we will provide a new update.

AirOn360 IoT Suite Pod IoT Suite Portal Degradation - NEW

This is to notify you about a Pod IOT Suite portal degradation. Some customers are complaining about not being able to log into the platform. This issue is intermittent and not affecting everyone. The engineers are addressing the issue with top priority, and updates will be provided as soon as available.