Mobile Network Comparison

Today, 16 March, has been a catastrophic day for Giffgaff – the mobile network has suffered a complete UK-wide outage that affects incoming and outgoing calls, texts and data services. The issues began at about 10am and seem to affect all users but not customers of their host network, O₂.

It appears that the issue was caused by a power failure at one of their facilities due to a burst water main. They may be well-reviewed, but questions will surely be asked about the competence of their logistical management given that this seems to be a single point-of-source failure and any redundant systems in place have not alleviated the problem. Giffgaff have been giving regular updates but they are quite hard to find (also due to this issue) so we’ll be keeping you up to date with all the latest news right here.

If you’re affected please keep checking back for the most recent updates:

Update – 10:27

I’m afraid that I can’t give you any ETAs at the moment, we are still investigating the issue. We will continue to keep you updated and I will be sure to post them as soon as we have any news.

Update – 10:38

We have not identified the source of the issue yet but we have narrowed it down. We will let you know as soon as we have further information.

Update – 10:49

We have discovered the cause of this outage is due to a power outage at one of our suppliers. We do not have an ETA on a resolution, but this will hopefully resolved soon. They will need to restore power, and then the engineers (who are already on their way there) will reboot all the required systems.

Update – 11:07

We can confirm that this issue will also make your balance and goodybags show as n/a in your account. You do not need to raise a case about this either as it will be fixed at the same time.

Update – 11:18

charlie_foster wrote:

why are we not allowed to raise a case? im just wondering

This is because this is a network wide problem, and does not need to be solved individually. Do not raise a case in regards to this service outage, as it will be resolved without your case being raised.

Update – 11:25

The reason we’ve not got this up on the main page (which we usually do in terms of wider issues with a “men at work” announcement) is that the system that we use to change this sort of thing on the site is affected by this outage too.

We’re working on getting all of this back up. As soon as we can, we’ll let you know – and of course, get this message up.

Update – 11:32

We have identified the cause of the power outage. A water mains pipe burst at our suppliers causing a power outage. Unfortunately we still do not have an ETA for you.

Update – 11:48

The engineers are still on the scene and are currently assessing the damage. We are expecting a call from them before 1pm to give us more information and we will of course update you as soon as we know.

Update – 12:08

We now have a “man at work” message on the login page. The message currently reads “The site is being updated at the moment so some of the transactional pages like login, top-up or activation may not be working. For more news on the update and timings check our community notice board”

It is showing this because we are moving our hosting in response to this issue. This is not a planned move and is in response to this issue.

Update – 12:31

Some incoming texts are working, some are not. If a text is shown on the senders end as sent but has not arrived on the recipients handset then it will be retried later. You will receive these messages after the issue is fixed.

Update – 12:57

We have had a conference call with the appropriate parties and an active plan has been designed and it is being investigated now. We will have more information in 30 minutes.

Update – 13:01

We have identified a possible workaround for voice only. More information on this coming up soon

Update – 13:46

Data should now be up and running for everyone. We are still working on the other issues.

Update – 13:58

Contrary to my previous statement, data should now be working for some members. Try rebooting your handset to get it working.

Update – 14:46

We have had an update from the engineers. Work is progressing and we are expecting another update within an hour to let us know how it is going.

Update – 15:20

There will be no issues with ports. Porting across the industry as a process do not happen on the weekend so unless it lasts that long, port ins will not be affected.

Update – 15:30

In addition to this I have just had news that we are unable to track the progess of some ports due to arrive today. If this applies to you then you need to wait until this issue is fixed before reporting this failed.

Mobile Network Comparison update – 15:38: We’ve noticed that voice calls seem to be working most of the time now but there’s still no official news from Giffgaff.

Update – 15:52

Data should now be working for all customers.

Update – 16:03

Engineers are implementing a workaround for voice services now. Voice services should be working for some customers and will continue to become available within the next hour or two for the rest. If your voice services are not working yet please be patient as the workaround is being rolled out across the country.

Update – 16:37

Many of you will notice that you are unable to login to post. This is because we have been running on the outages server in order to limit transactions as they currently fail on live.

Due to maintenance performed by lithium all members, including staff, were logged out and since we were on outage servers nobody, again including staff, were unable to log back in. We briefly switched back to live servers in order for staff to log back in, but we should very soon (at time of writing) be back on outage servers again, so this is why you cannot log in.

Update – 17:03

As mentioned before, data and voice services are being brought online for everyone, but in stages. You will notice that some members are getting it back before others; this is completely normal and to be expected. You should expect to have your data and voice back soon if you have already.

We do not have an ETA for when texting will be back up.

Update – 17:30

Just to reiterate what I said earlier; many of you will currently be unable to log into the forum. We are aware of this as unfortunately this is an unavoidable side effect of the measures we have taken to prevent people from performing transactions.

We are aware that many people will continue to view this thread so we will continue to post in it and keep you updated even as the number of posters dwindles.

Update – 18:05

We apologise for the downtime on the forum. Again this was due to us being logged out while running on the outage server.
Voice and data services have been returned, but these will be intermittent. We are continuing to work with the engineers to ensure at the highest level to ensure that a full fix is reached.

Update – 18:41

Base Services

I am very happy to confirm officially that all basic service (calls, text and data) should now be mostly back up and running.

There is a chance that these could still be intermittent for the time being, so there is still no need to raise a case. Teams will continue to work on this this evening and we will continue to provide updates for you.

Account Services

However for account based services (porting in, activating, topping up, buying a goodybag etc.) there may still be a backlog or delay so I must urge you to refrain from trying these for the time being. Again, we will provide updates on this where we can.

Ports due to complete today

These will have been delayed and we are looking for an update on these now.

Update – 18:50

We are sorry for the inconvenience this may have caused you. As we’ve updated in this thread, our tech teams have been working on this and have put a first fix in place which we are now monitoring for stability. During the period where service is restored you may notice that the service is intermittent. We will continue to work on this issue throughout the weekend if necessary. Regular updates will continue to be provided here as and when we have them.

At the moment we’re focusing on restoring service for you all, but we want to reassure you that once this is done we will be looking at ways to make this up to you.

Update – 19:02

I have just had confirmation that top ups and goodybags should now be working and you should be able to purchase new credit and goodybags.

Provisioning is possibly still affected however, meaning that activating and ordering new SIMs might still have issues and so you should refrain from ordering new SIMs or attempting to activate for the time being.

Update – 19:03

We now have a blog up about this. You might like to read it – http://community.giffgaff.com/t5/Blog/Today-s-Service-Outage/ba-p/3432147

We are aware that there is an issue with the giffgaff network including but not limited to loss of service. We are aware of this issue, please do not raise an agent case.

We are investigating the issue and will update you as soon as we have more information.

Update – 23:26

Thank you all for your patience today – if you haven’t already, please see our apology here. Please be assured we are continuing to work on this issue and will provide regular updates on this thread.

All phone services are currently up and running now as normal, if this isn’t the case, please try a reboot and hard reset.

Ports: We have a number of members who were due to port in today. At the moment we are waiting to talk to our third party company about their statuses and when we can hope to have them completed. We will update you as soon as we receive an update ourselves.

Activations should now be running through fine – please don’t be alarmed if you experience a slight delay, however upon monitoring the situation, they are happening as usual.

Account Services: are also running as normal and there should be no delay on services such as topping up and buying goodybags.

Next Official Update: 10:30am tomorrow morning.

Regardless of official updates, the community team will continue to pick up on individual posts from 8am tomorrow morning.

At the moment we’re focusing on restoring service for you all, but we want to reassure you that once this is done we will be looking at ways to make this up to you. Thank you once again for your patience, your comments and understanding.

Update – 08:49 – 17/03/2012

This update only concerns those who have experienced a loss of balance since the loss of service yesterday.

If you have lost balance since the loss of service yesterday and you think that this is not due to normal charging then you need to raise an agent case with the title “Lost balance after service issue 16/03/2012”.

If you have a different problem that is still unresolved then please do not raise an agent case yet.

Update – 10:50 – 17/03/2012

Port ins

This only regards members who attempted to port in yesterday and are now still without service. These are being processed now and should be completed by 3pm today. It may take up to an extra hour for balance to show for these accounts however. Please do not raise a case about port ins that failed yesterday.

Queued Goodybags Overnight

The majority of these were successful, we are now working through these manually and hope to have them all completed by 3pm. Please do not raise an agent case if your goodybag is still not working.

Port outs

Port outs may have issues until Monday morning when the networks that the members ported to will be able to pick up the files we have now sent them. Please do not raise an agent case about port out issues.

Activations

All activations should now be working successfully. If you are having an issue with an activation that is taking an unusual length of time or has failed then please do raise an agent case.

Final Update – 08:59 – 18-03-2012

This issue is completely resolved – both the core cause and the lagging effects, i.e. affected port ins, slow activations,

If you are experiencing any loss of service please check your APN settings are correct and perform a reboot. If the issue persists, raise a case with our agents who will investigate.

We will be posting more about what happened at a later date when we fully understand the outage; cause, effect and how we will be making it up to our members.

Once again, thank you for your patience and understanding over Friday and Saturday,

Sorry for the inconvience,