vowe.net Status

Harddisc Seagate Barracuda 1500GB

While vowe.net is offline, a bit about what is going on. The site used to run in shared hosting at 1&1. When that broke, Stefan let me host on his server. The site got faster, much faster actually, and he added SSL via LetsEnrcypt.

Once in a while the site crashes. When that happens, Stefan has to restart it. If it happens at night or while he is working, it may take a few hours, but then it’s coming back. I like to phrase it as “vowe.net is taking a bath and will come back squeaky clean”.

Well, not this time.

The server hardware is failing. More precisely, the storage is failing. It’s a simple RAID 1 setup. Two drives, one holds a copy of the other. When a drive fails, which does happen more often than you think, you just replace it, the data gets synced back and everything is back in good shape.

Well, not this time.

While the RAID was rebuilding, the other drive failed as well. Again, this is more common than you might think, especially if you have Seagate Barracuda drives. (Hint: not recommended). Rebuilding puts a lot of stress on the old drive, and this time around it failed. If you followed along, that means catastrophic failure. Everything is lost.

Not, if you are Stefan.

Stefan has been doing server farms for more than a decade, on a very large scale. When a RAID drive fails, you don’t replace it. You take it offline so that nothing writes to it. Then you make a fresh backup. Only then do you start replacing the failed drive and rebuilding the RAID.

That’s where we are. Both redundant drives failed. Both have been replaced. The RAID is being synced. Next steps: reinstall and restore. That sounds easier than it is, if you are already living a 25 hour day.




2 thoughts on “vowe.net Status

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s