The 5DH Blog

Navisite virtual servers went down overnight and have stayed down for upwards of 9 hours and counting with some systems not yet restored.  Sound familiar?

How do we know about this?   We’re still on their NOC update e-mailing list for some reason.  Let me tell you, it’s nice to receive these messages (such as “we estimate another 30 minutes for full restoration” and “our engineers are working on the problem and estimate 30 minutes to full resolution” - received over 2 hours ago) and not be affected at all by them.  Just in case anyone was still wondering if all the trouble we went through was worth it or not….

Craig

April 4

We’re making progress on the daunting data-entry task (putting everything into the new billing system by hand).  We’re probably around 60-70% done so there is a light at the end of the tunnel…

We’ve added some new URLs: clients.5dollarhosting.com and billing.5dollarhosting.com both go to the new billing system and are easier to remember.  And order.5dollarhosting.com is live too, for new orders.

I updated a bunch of the billing system templates today as well as our home page.  Much work remains on our site, but piece by piece it will get done.  Good news: semi-automated ordering is now back online!  (We still hand-verify all new orders).

Terry, the kids & I are off on a MUCH-needed vacation next week.  We’ll be checking in every day, don’t worry.  Now, if I can only manage to get our taxes done before we leave… *sigh*

Craig

This one had been harassing us since the switchover: user goes to send email via WebMail, gets a 535 error ‘Authentication Failed’.  This was affecting both SquirrelMail and Horde users.  The fix, as listed in our forum, said to force a password reset.  Well, that worked, but sometimes had the “unintended side-effect” of rendering the mailbox in question invisible in the user’s cPanel.  So we got a ticket anyway, since we have to fix that ourselves on this end.

Good news!  We believe we have permanently eradicated this problem.  For those technically inclined, read on for what we discovered.  Everyone else: just know that we don’t think you’ll be seeing any more of this particular buggie.  But if you do - let us know
by opening a support ticket.

THE PROBLEM
Users cannot send mail with WebMail.  They gt a 535 error

THE CAUSE
Incorrect group ownership on 2 files within user account (/home/$user/etc/$domain/passwd and /home/$user/etc/$domain/shadow)

PREVALENCE
This appeared to affect only email accouts for users who were ‘imported’ from the old servers (which of course is most of them).  Newly-created accounts did not experience this.

FIX
Change group permissions on the files to be owned by ‘mail’ group (note: we pro-actively did this for everyone on all servers).

Here’s hoping it stays squashed - it was awful hard to track down!

Craig

A client sent this in to us, it’s today’s Dilbert.  How strange, it seems vaguely familiar… 

I know I haven’t posted in a while. The last 30 days were extraordinarily busy for us as we all got used to the new servers. Now that things have calmed down a bit I took some time to work on the blog today, with the goal of making it more useful to you. I have enabled commenting (with serious spam protection in place!) so that you can voice opinions here as well as in the forum. I will try to update this more frequently going forward.

Thanks,

Craig

If your work environment has a firewall which is preventing you from accessing your cPanel and/or WebMail, and your IT guys are not willing to open the necessary ports for you, see this post for a possible solution

http://5dollarbackup.com/board/viewtopic.php?t=360

Craig

POP method (not 100% guaranteed to work, but worth a try!)

http://5dollarbackup.com/board/viewtopic.php?t=338

WebMail - exporting and importing to Horde:

http://5dollarbackup.com/board/viewtopic.php?t=337

 NOTE: for both of these, you need your old account IP address.  We can provide it if you need it, or if you can find your original Welcome Letter and save us a ticket we would be very grateful!

NOTE 2: We do not know for how long the old servers will remain accessible.  This is something to try sooner than later.  I am making backups as frequently as possible, and should the old servers go away before you’ve had a chance to check - we do have a plan B.  We can in fact convert the backed-up mail into Horde, but it is time-consuming.  So if you can get it now, great.  But do NOT pull out your hair worrying that you have lost mail.  It’s in one of 2 places, and we have both. 

Thanks,

Craig

Folks, if the response time is a little longer than normal it’s because every time we answer a ticket 2 more pop in.  I suspect it will keep like this for a while before it dies down.  We appreciate your patience.  If you have a ticket in and haven’t heard from us - we haven’t forgotten you, I promise.  Check the forum as well, as some of our Early Adopters are hanging out today and can give you pointers as well.  I won’t be able to spend much tims there until we get the fires put out.

I just ran what will probably be the last RSYNC from the old servers.   If you’re missing a file or see old content, we’ve got the latest stuff, just ask.  Hang in there, we’re almost done with this mess.

Craig

A quick note as we’re literally drowning in tickets.  If you read this and you missed the other notices about the server transition, here are the things you don’t need to put in a ticket for:

- Email doesn’t connect/you don’t get it or send it.  See:  http://www.5dollarhosting.com/newserver-email.htm  for all the stuff you need to tweak your email software and/or the new WebMail

- Email receives, but you can’t send using a client OR WebMail!  You get a ‘535 Authentication Required’ message.  See: http://www.5dollarhosting.com/newserver-email.htm  (force a passwor4d reset in cPanel, problem goes away).

Due to the EXTRAORDINARY amount of support requests in the queue, if you are in one of the above situations it will likely be much faster for you to quickly make the necessary tweaks and be all set than to enter a ticket.  We appreciate your patience while our crew works through the pile.  (Second only to a certain weekend in November for volume…)

Craig

That was the thought going through my mind just a few minutes ago as I completed typing the commands that finalized the changeover of the primary and secondary nameservers.  I had a lot of trouble hitting ‘enter’, I’ll tell you that.  A few keystrokes among billions over the past 8 weeks, yes - but probably the ones that meant the most and caused the most trepidation.

That means, in non-geek English: the cord is cut.  Everyone should begin resolving shortly to the new servers and their new accounts (ready or not, here we come)!  I expect to hear from folks who didn’t have a chance to check their sites or aren’t clear on the new email settings, etc.  We’re ready for you.  We will fix it if it’s broke.  Most won’t need us to.

We’ve increased the frequency of data retrieval from the old servers to 4 times a day each. That means if you look at your site in the morning and it’s got old stuff on it, let us know.  We’ve got the new stuff tucked away and can easily copy over anything you want us to.

The old servers (and your old accounts) will remain available by IP address, but for how much longer we don’t know.  Our best estimate would be anywhere from 9 AM through the end of the day on 1/2/08.  Could be later, we know how quickly the folks at (you-know-where) move in general.  Although Murphy’s Law would say this is the one time they act with lightning speed.

The end of an era?  It’s hard to get sentimental about nameservers - they’re just settings, really.  But it still tore at my heart to flip that switch.  I remembered how proud and excited we were back in May 2000 when we created them.  I sure hope they (and you) like it in our new home.

Craig