Page 1 of 2

MOUL Fixed!

PostPosted: Sat Jan 26, 2008 8:44 am
by andylegate
Hey guys, just thought I would mention this.

As some of you may or not have been aware of, there were a lot of people having problems logging on to MOUL.

Problems ranged from very long log in times to not being able to log on at all.

Well Cyan has finally found and fixed the problem!

Turns out that when they moved the servers, the servers got an update. This update was interfering something with the TCP. This caused problems for some, and some of those problems were bad.

They've put in a temporary fix, and as one of the aflicted, I can now log in to MOUL in under a minute again, instead of 6 to 15 minutes!

Cyan promised to do a more perminat fix when they get a chance. Here's the thread talking about it over in the MOUL forums:

MOUL Fixed

Re: MOUL Fixed!

PostPosted: Sat Jan 26, 2008 11:49 am
by Chacal
Doh!
They really must be lacking resources, because their sysadmin should know The Rule:

You never, never, never put an upgrade in production without testing it in a lab first.

In extreme cases, when you must do it, then still don't do it.
Got absolutely no choice? Still don't do it.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 7:57 am
by D'Lanor
Chacal wrote:You never, never, never put an upgrade in production without testing it in a lab first.

Who says they didn't? Since this only affected a tiny percentage of players a small scale test would not have revealed the problem.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 2:07 pm
by Chacal
Maybe they did. Who knows? Honestly I doubt it.
It is still good advice though.
Here's another: don't change too many things at once.
First move the servers, let them run for some time, if all is well then upgrade the OS. Or vice-versa.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 2:27 pm
by Nadnerb
A) The move was precipitated by the closing of the original data center. They pushed the move as far up against the deadline as possible in order to finish season 1 on the original servers.
B) They did not "move the servers". The server software was transplanted onto supposedly identical machines at the new data center.
C) As D'lanor said, testing on any scale besides the full userbase would not have revealed the problem, due to it's nature, causing problems only in specific network configurations and conditions.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 2:37 pm
by andylegate
Cyan also said that the SP2 update happened automatically after the server move too. The way they explained it, it was not something they themselves implemented.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 7:28 pm
by Chacal
Nadnerb wrote:B) They did not "move the servers". The server software was transplanted onto supposedly identical machines at the new data center.

That's what I mean by "moving the servers".

Nadnerb wrote:C) As D'lanor said, testing on any scale besides the full userbase would not have revealed the problem, due to it's nature, causing problems only in specific network configurations and conditions.


It would have made sense to test it with the full userbase in the production environment. At least it would have been a controlled test with an easy rollback possibility.

andylegate wrote:Cyan also said that the SP2 update happened automatically after the server move too. The way they explained it, it was not something they themselves implemented.


Yeah this one sucks. I would be quite pissed off at the provider for that one.

Re: MOUL Fixed!

PostPosted: Mon Jan 28, 2008 10:47 pm
by Grogyan
Chacal wrote:
andylegate wrote:Cyan also said that the SP2 update happened automatically after the server move too. The way they explained it, it was not something they themselves implemented.


Yeah this one sucks. I would be quite pissed off at the provider for that one.


Just had a gathering in AH hood tonight, and many players now see many things broken now since the fix, than before, sadly no fix yet for broken marker games.

I hate Windows, and I despise Billy enough to whollop him one, even if it means getting arrested.

Re: MOUL Fixed!

PostPosted: Tue Jan 29, 2008 7:57 am
by D'Lanor
andylegate wrote:Cyan also said that the SP2 update happened automatically after the server move too. The way they explained it, it was not something they themselves implemented.

Turn off automatic updates? ;)

Re: MOUL Fixed!

PostPosted: Tue Jan 29, 2008 11:01 am
by Chacal
OMG you're probably right, they kept automatic updates enabled.
I'm no hosting professional, but even I thought of turning that off on my servers.

I think I would change providers.