Wasn't our fault this time
Some time earlier today our hosting company (PowerVPS) dropped off the Internet along with it's sister companies, and all their hosted VPSes (us included).
I noticed around lunchtime (when I've been keeping an eye on the forums; I'm officially back now BTW... BT got their act in gear for today) and then through until around 8 o'clock UK time there was nothing, then between 8 and 9 the VPS came back, and dropped again, and came back, and dropped again (and so on, ad reductum)
It's still not 100%, the response times are "interesting" at present, but it's getting there slowly but surely, and I'm hoping that PowerVPS (or Virtacore, the parent company) will have an explanation sometime soon...
OK, Virtacore had the following in their news (PowerVPS are still "off-net" it seems...)
29 May 13
Network Outage
Update: 5/29/2013 – 3:21 PM Eastern Standard Time
Virtacore would like to apologize to all of the existing customers. We wanted to provide you with latest information on our network issues:
One of Virtacore’s primary network providers is experiencing a failure within their core infrastructure. Virtacore is trying to re-advertise the IPs that are impacted at a higher priority. This is being done to bypass the network provider that is having the problems. We are doing this as we work with the provider to resolve the issue. According to our network provider this is something they have not experienced so it is difficult for them to give us any firm estimates of when the issue might be fixed.
Our engineering team is working as quickly as possible to resolve this issue on many fronts. As soon as we know more, we will keep you informed.
Thank you again for your patience.
Previous Post (1:47 PM):
—————————
We are still working on the issue. We have been asked about our redundant network providers. Our redundant network providers are currently functioning without problems. Which explains why we have many of our customers functioning without issues. Unfortunately, we are having difficulties separating the traffic currently routed through that provider. So we are working on ways to fix the situation independent of the network provider trying to resolve the issue.
We will keep you posted on time frames as soon as we provide one with certainty.
We apologize for any troubles this has caused our customers.
Previous Post (12:59 PM):
—————————-
We apologize for the continuing issue with our network. We have learned that our network provider is currently trying to solve a problem with the routes looping in their network. We have raised this issue to the highest level of escalation with the company and they have informed us that they are working aggressively on the issue. This is causing connectivity issues for some of our clients, intermittent issues for others and some customers are not experiencing issues at all.
Unfortunately, our network provider has not given us a firm time frame of when we can expect a resolution. We are looking into our redundant providers to minimize, and potentially eliminate, the problem as we continue to wait for the network provider.
Again, we would like to underscore the fact that the data and systems are in no way compromised or impacted by this issue.
As soon as we have more information we will post it immediately.
Previous Post (11:33 AM):
—————————–
Some of our customers are still experiencing issues. We are working with the network provider to find out why this issue is causing additional problems. Our technicians have identified this as a routing issue with the network provider.
Our network provider has not given us a definitive time window of when they will be able to resolve this yet. We are waiting on additional information from them about the nature and depth of the problem and how to resolve it.
Previous Post (10:22 AM):
—————————–
This morning between 6 AM and 7 AM Virtacore experienced an issue with our network provider and DNS services. As a result some of our customers experienced trouble with access to hosting environments. The systems and data were not impacted by this issue. All systems and data were up during the period of the issue.
During the network outage we had issues with our own phone, email and website systems. As a result Virtacore had a limited ability to communicate with customers.
Our technical team is working with our network provider to completely correct the issue, and prevent this from occurring again.
As soon as we have more information, we will be contacting our customers directly.
We apologize for any inconvenience this may have caused to our existing customers.
Thank you for your patience.
Virtacore
