Saturday, February 27, 2010
YeeeeeeeeeeHaaaaaaaaaaaaaaaaaaaW!
I got up this morning expecting the same... can't connect to the server.... instead it was a nice surprise to see tbca.org pop up... my next thought was, okay can I connect and update - yes!! So I updated the site and we are back to "normal" again... I'm going to leave this site up for the simple reason - it could happen again - hope not - but if it does everyone will have a place to get some info... sorry for the down time but that's life!
Friday, February 26, 2010
Light at the end of the Tunnel?
We'll see, but they tell me I should be back online and able to update by tomorrow around 3pm... we'll see...
Get your D1 scores and stories here: D1 Baseball Scores
I'd be remiss if I didn't mention my alma mater WKU beats Texas A&M 5-1 as BGA product Matt Ridings gets the win!! go Toppers!! ah, those days on the hill...
on the homefront - Belmont, ETSU, UT-Martin, and Austin Peay are already in the win column... Trevecca sweeps Bethel (IN), Freed Hardeman, Martin Methodist, Union, and Lee all won today... see scores here: NAIA Scoreboard
Get your D1 scores and stories here: D1 Baseball Scores
I'd be remiss if I didn't mention my alma mater WKU beats Texas A&M 5-1 as BGA product Matt Ridings gets the win!! go Toppers!! ah, those days on the hill...
on the homefront - Belmont, ETSU, UT-Martin, and Austin Peay are already in the win column... Trevecca sweeps Bethel (IN), Freed Hardeman, Martin Methodist, Union, and Lee all won today... see scores here: NAIA Scoreboard
FRUSTRATION SETS IN...
I am very frustrated right now... my hosting company has been very helpful, but so far no results as far as getting the site back online... it has been 4 days+ now... after chatting online to them last night I was hopeful as the guy told me it should be good to go around midnight last night - I got up this morning and boom... no site... If you are reading this spread the word - I am updating best I can without my usual resources.. and when it's finally online and viewable again you'll have a lot to catch up on.... sorry guys!
Thursday, February 25, 2010
Thursday Feb 25 Afternoon UPdate
okay - the server the tbca site is on is back up and they are restoring accounts - it appears that this should be complete by about noon tomorrow... as of 4:20 our time there were 48 accounts still left to restore... I'm trying to get them to help me repoint the site to a new server right now and I should be able to get the site up tonight... however I am having a hard time getting in contact with them - if I could figure it out I'd do it on my own but I'm not that smart - I am trying though...
here's a post from their site pertaining to the server we are on:
here's a post from their site pertaining to the server we are on:
| wsl02013 | Shared | 02:19pm MT 2/25/10 - 48 Accounts remaining 01:07pm MT 2/25/10 - 71 Accounts remaining 12:07pm MT 2/25/10 - OS Reload complete, accounts are restoring now - 97 accounts remaining. We will then need to link the client accounts and the server will return to normal service. |
We're almost back online... Latest
I have the site republished, but (and this is tech speak, sort of) it is pointed toward the old servers (which are still down) so they (Westhost) is in the process of getting them redirected - I am hoping we will be back online by tonight....
Monday, February 22, 2010
Latest on Outage
Dear WestHost Client,
Thank you for your patience during this very difficult time. As you well know, WestHost has a history of delivering solid and reliable hosting platforms with top-notch support. We are not perfect by any means, and recognize areas where we need to improve, but we aim to please and strive to deliver real value to you our loyal customers.
At present, we have restored service to all but 12 shared and 6 dedicated servers. To our great concern, your account is residing on one of those servers currently being impacted by this outage. Our team of Systems Administrators and Engineers are continuing their work to restore the remaining servers and have been working non-stop for over 48 hours with little or no sleep. We continue to place the highest level of urgency on restoring these remaining servers. This is our number one priority at all levels within the organization.
Now for some details:
As a result of the fire suppression system error which was triggered at the Data Center during a routine system check by one of the fire system vendors, critical hardware components were damaged across dozens of our servers. Our first course of action was to replace those damaged components. We were able to repair or replace almost every failed hardware component during the first 36 hours through a combination of our on-hand hardware supplies and an expedited hardware shipment from our primary supplier.
Unfortunately, many of these hardware problems impacted hard drives, causing hard drive failures. When this happens, data loss becomes an immediate focus. Retrieving lost data and restoring from backups can take up to 24 hours per server. This is where we are currently with a majority of our downed servers. Our backup process restores three servers simultaneously. We will get to each server as quickly as we can, but given the number of servers needing to be restored from backups, this places our ETA anywhere from 24 to 96 hours. In other words, it could very well be Friday of this week before all servers and accounts are back on line. If ANYTHING changes with regard to this timing (good or bad) we will immediately inform you. We profusely apologize for the .stops and starts., however, once one issue was resolved another problem surfaced which resulted in adjustments to the published ETA.
In addition to the failed hard drives, we have also found other dedicated and shared servers that are experiencing file system problems. While accounts can generally function on these servers, file system problems can negatively impact the performance of the server. We are aware of these servers and are tackling these issues as they become evident.
What does this mean to you?
Due to the length of this restoration process, we are currently preparing new servers in our WestHost 4.0 environment for you to get your site back up sooner if needed. These new servers will be ready for clients to start using within the next few hours. As soon as they are ready, we will send you further details. This free solution will temporarily enable you to get your email and a basic site up and running so you can communicate with your customers. We recognize the impact this has had on your business and we plan to offer compensation via free month(s) of service for your loss. Once again, we are 100% committed to doing what we can to restore your business and confidence in our company going forward.
For those who might have a question about the Data Center facility, it is important for you to note that we are co-locating at this location which is a Tier 4 data center managed by some of the best professionals in the industry. It has been fully operational throughout this entire process. Most of our systems are running perfectly fine, without any residual effects. Power, bandwidth, cooling, and other critical systems are working properly. This issue has purely been a result of a fire suppression error, and not as a result of any mistake by WestHost personnel or systems.
We will continue to provide email updates every few hours. Again, thank-you for your patience as we restore your services.
Sincerely,
Jeff Hunsaker
President/COO
UK2 Group, US Operations
WestHost Brand
Thank you for your patience during this very difficult time. As you well know, WestHost has a history of delivering solid and reliable hosting platforms with top-notch support. We are not perfect by any means, and recognize areas where we need to improve, but we aim to please and strive to deliver real value to you our loyal customers.
At present, we have restored service to all but 12 shared and 6 dedicated servers. To our great concern, your account is residing on one of those servers currently being impacted by this outage. Our team of Systems Administrators and Engineers are continuing their work to restore the remaining servers and have been working non-stop for over 48 hours with little or no sleep. We continue to place the highest level of urgency on restoring these remaining servers. This is our number one priority at all levels within the organization.
Now for some details:
As a result of the fire suppression system error which was triggered at the Data Center during a routine system check by one of the fire system vendors, critical hardware components were damaged across dozens of our servers. Our first course of action was to replace those damaged components. We were able to repair or replace almost every failed hardware component during the first 36 hours through a combination of our on-hand hardware supplies and an expedited hardware shipment from our primary supplier.
Unfortunately, many of these hardware problems impacted hard drives, causing hard drive failures. When this happens, data loss becomes an immediate focus. Retrieving lost data and restoring from backups can take up to 24 hours per server. This is where we are currently with a majority of our downed servers. Our backup process restores three servers simultaneously. We will get to each server as quickly as we can, but given the number of servers needing to be restored from backups, this places our ETA anywhere from 24 to 96 hours. In other words, it could very well be Friday of this week before all servers and accounts are back on line. If ANYTHING changes with regard to this timing (good or bad) we will immediately inform you. We profusely apologize for the .stops and starts., however, once one issue was resolved another problem surfaced which resulted in adjustments to the published ETA.
In addition to the failed hard drives, we have also found other dedicated and shared servers that are experiencing file system problems. While accounts can generally function on these servers, file system problems can negatively impact the performance of the server. We are aware of these servers and are tackling these issues as they become evident.
What does this mean to you?
Due to the length of this restoration process, we are currently preparing new servers in our WestHost 4.0 environment for you to get your site back up sooner if needed. These new servers will be ready for clients to start using within the next few hours. As soon as they are ready, we will send you further details. This free solution will temporarily enable you to get your email and a basic site up and running so you can communicate with your customers. We recognize the impact this has had on your business and we plan to offer compensation via free month(s) of service for your loss. Once again, we are 100% committed to doing what we can to restore your business and confidence in our company going forward.
For those who might have a question about the Data Center facility, it is important for you to note that we are co-locating at this location which is a Tier 4 data center managed by some of the best professionals in the industry. It has been fully operational throughout this entire process. Most of our systems are running perfectly fine, without any residual effects. Power, bandwidth, cooling, and other critical systems are working properly. This issue has purely been a result of a fire suppression error, and not as a result of any mistake by WestHost personnel or systems.
We will continue to provide email updates every few hours. Again, thank-you for your patience as we restore your services.
Sincerely,
Jeff Hunsaker
President/COO
UK2 Group, US Operations
WestHost Brand
Site Outage Update
Here is the info I have on what's going on:
There were multiple servers impacted by this outage; some of the servers have had damaged hardware replaced while other servers need to be restored from backups. A large majority of the servers that were affected have since had all services restored and are back online. We still have a few remaining servers that are experiencing problems. We are doing everything we can to restore those remaining servers as quickly as possible.
We apologize for any inconvenience or frustration this outage has caused you. We are committed to providing you with the highest level of service possible. We will be providing more information on your server status throughout the day. Thank you for your patience while we strive to improve our services. Please contact us if we can be of assistance.
Hopefully they'll get us back up in the next 24 hours
Feb 22, 9:00 a.m.
On Saturday, February 20th at approximately 2:20 p.m. MDT, the server your account is on experienced some hardware failure as a result of an annual fire system inspection at our Data Center (DC). An inadvertent release of Inergen was triggered in the DC environment. The release was triggered by an actuator that was not removed as required in the DC's pre-test checklist. As such, the entire DC was impacted at some level, however not all servers were affected. This error was not the result of a mistake by WestHost employees, our hardware or systems. The mistake was triggered by an error from a vendor. With that said, we take responsibility and are doing everything we can to restore all systems completely.There were multiple servers impacted by this outage; some of the servers have had damaged hardware replaced while other servers need to be restored from backups. A large majority of the servers that were affected have since had all services restored and are back online. We still have a few remaining servers that are experiencing problems. We are doing everything we can to restore those remaining servers as quickly as possible.
We apologize for any inconvenience or frustration this outage has caused you. We are committed to providing you with the highest level of service possible. We will be providing more information on your server status throughout the day. Thank you for your patience while we strive to improve our services. Please contact us if we can be of assistance.
Hopefully they'll get us back up in the next 24 hours
Sunday, February 21, 2010
Blogging - every now in then it comes in handy...
For the first time since I started using Westhost to publish my website (about 8 years) they have had their first major disaster... my site has been down since yesterday around 7pm... can't update - I actually have the Saturday update completed, but I can't publish it... I'm waiting right now on a live chat with Westhost to see when they think this thing will be repaired and am hoping that some of you will see this - I plan on emailing coaches to make them aware, but also - keep send me info as you have.... We got some great weather this weekend but it appears to be short lived as rain is pounding my roof and we may even have some more snow... speaking of snow I have some great pics of some of my friends and family, some former players of mine, sledding with their kids over at my other blog for my photography work... The guys on the tube - that's Chris Edwards (Beech '96) on the right - that kid hit one of the longest HR's I've ever seen at a tourney in Lincoln Co. - they called it foul, but I don't know how - it was high, long and quickly out of the yard - he got rooked... he went on to play at Vol St and USC-Aiken... the other guy is Grant Edwards (no relation to Chris/ Beech 2000) Grant builds houses for a living.... OnDeck Sports Photo Blog
Subscribe to:
Posts (Atom)