Monday, October 07, 2013

Case For Humanity Kickstarter

A couple of months back I decided to design a laser-cut wooden box to hold all of my cards from "Cards Against Humanity".  After making one, I realized it would also work nicely for other card games like Magic: The Gathering, Dominion, etc.

I also recently ordered a laser cutter to help with some of the other projects I've been designing and selling.  So, I wondered what would happen if I posted my card box on Kickstarter to help earn funds to pay for this laser cutter.  Anything I made would help, so I set the funding goal to $1,000 dollars.

We hit that in 1 hour!

Thirty days later, my project had 600 backers and over $34,000 in pledges!  I had no idea that something as simple as a box to hold cards would be so popular.  Turns out lots of people like custom gaming accessories and this simple box filled a need.

The project is now over, but its not too late to get in on the action.  If you're interested in a box or kit, you can sign up using the link below and you'll be put into the queue to receive one after the other rewards are fulfilled.

Thanks to everyone who participated in the project and anyone thinking about ordering a box! 
- Greg


Sunday, April 08, 2012

The Brownie Rule

The brownie rule states that when brownies (or any treat of similar nature) are cut, they must be cut of uniform size.  The brownie rule does not prohibit one from eating more than one brownie, but when a brownie is taken from the pan, it must be cut to a size that conforms to a uniform grid pattern.

One may cut a brownie that is larger than the minimum dimension as long as the overall dimensions are multiples of the minimum dimension.  At no time may the main brownie mass be cut to a size that doesn't confirm to the grid.  This can lead to confusion for future brownie cutters and often results in a chain of violations.

Grid Size
The grid size is determined by the baker of the brownies.  If the baker fails to establish a proper grid size verbally or by failing to make the first cut, he/she forfeits the the grid size decision to the first person to make the cut.


Smaller Portions
Brownies may be eaten that are smaller than the minimum dimension as long as a piece meeting the minimum dimension is severed from the main brownie mass first.  Once cut to the minimum dimension, the piece may be cut into smaller sections and distributed as you see fit.


See Figure A for examples of valid and invalid brownie cuts.

Figure A



Note that the brownie rule may also be applies to cakes, pies and any such treat that may be cut to even sizes.  As a courtesy, you should always make you guest know when the brownie rule is in effect.

Wednesday, February 08, 2012

Moving on to the "next" great thing!

After more than 5 years working on the EverQuest II development team, it's now time to move on to the next great project!

I've been an avid player of the EverQuest franchise since 1999. I was even in the beta for both EverQuest and EverQuest II.  EQ is the reason why I'm working in the game industry.  I've always loved computer games and loved programming, but if it weren't for my passion for EQ, I would have never made the 1800 mile journey leaving family and friends to come to San Diego and join SOE.

I first met the EQII dev team at Fan Faire Atlanta in 2006.  I participated in a "Dev Brunch" where I sat with members of the team (specifically Josh "Autenil" Kriegshauser") and got to pick their brains for an hour.  While many other fans were offering opinions about the game and voicing their complaints, the only thing I wanted to talk about was what happened behind the scenes.  I asked them everything from code questions to hardware configurations and many times I got the same answer.  "Sorry, we can't really talk about that."  I was excited and disappointed at the same time.  These guys were doing what *I* wanted to be doing and I couldn't get enough!

 My wife and I (on the right) with some of our guildmates at the 2006 Fan Faire.

When the brunch was over I chased down Scott Hartsman, the EQII Producer at the time, and expressed my interested in applying for a job and getting on the dev team.  Of course he was really busy and probably got a lot of half-serious requests like that, so I didn't really expect it to go anywhere.  During the remaining days at Fan Faire, I made some other friends with dev team members and spent the most time with Eric Swanson.  Eric was also a very active player at the time and we talked a lot about the game.  When I returned home Eric moved to my server and joined my raiding guild.  We stayed in touch almost daily and a few months later he sent me a message in game.

Eric: "Hey, you still interested in applying for a position here?  We're looking for a coder."

Me: "Hell yeah I am!"

Ok, so I don't remember the exact conversation, but it was something like that.  Within a few days my first phone interview was scheduled.  I spoke with Josh who was one of the senior programmers at the time and remembered me from Fan Faire.  Admittedly I didn't have a lot of experience in C++ but had been programming professionally for about 12 years and longer if you count the things I did in my own time.  Despite that, I thought the phone conversation with Josh went well.  A day or two later I was scheduled to talk to Don Neufeld who was the Tech Director at the time.  He was a tough interview but I still thought things went well.  That was until I received an email a couple of days later telling me that they appreciated my time but didn't think I was a good fit for the team at the time.

To say that I was disappointed was an understatement.

Not being the type of person to give up and take no for an answer, I asked Don for advice.  Where did he think I needed work and what could I do to gain the experience they needed?  He gave me some good advice and pointed me at a few good C++ related books which I quickly purchased and read over the next few days.

A few days later, trying my best not to beg, I asked for another chance at the position.  This time, Josh got back with me on a Friday with a written test that he wanted me to take and give back on Monday.  There were a lot of tough questions and code projects on the test, definitely some things I had never done before.  I spent the entire weekend working on it and gave it back Monday.  It was the hardest few days waiting to hear a response.  When they got back to me, it was good news; they wanted to fly me out for an in-person interview.

It wasn't long before I was stepping off a plane in San Diego.  I had never been there before and the beauty of the city coupled with the excitement of visiting SOE headquarters after having played EverQuest for the past 7 years left me beside myself.  If you've flown into San Diego during the day you know what I'm talking about!

Downtown San Diego and the U.S.S. Midway

My in-person interviews went well.  I like to think I'm a pretty personable guy and I think it really helped my case to talk to them in person.  They were in a position where they could really use someone to work on the EQII tools and since my background was primarily in "managed" code, they asked if I'd be willing to work on tools if they offered me the position.  I was thinking, hell, I'll clean the bathrooms if you offer me the job!  I was really impressed with SOE and my visit there made me want the job even more.

A few days after returning home I received an offer letter!  The only downside was that it was a 40% pay cut from my current job and that didn't include the increased cost of living going from Tennessee to Southern California!  Even so, it didn't take a lot of thought to know that I wanted to take the job.  It took a little convincing to get my wife to agree to leave our family and friends, but she was very supportive of me and understood that this opportunity meant the world to me.  I accepted the offer and agreed to start 30 days later.

During my last month in Memphis we packed our house, put it on the market, returned to San Diego to find a place to live and shut down two other businesses that I was running at the time.  It was a huge amount of work to do in a month but the entire time I was looking towards the adventure to come.

In my 5 years on the EQII team I worked my way up from a Tools Programmer to a Senior Developer and eventually the Lead Programmer.  I still don't profess to be the most intelligent or skilled programmer at the company or even on the team.  I worked with lots of immensely intelligent people from which I learned a lot.  But I like to think that my hard work and dedication got me where I am today.

The EQII team raising our flag over the company headquarters (2012).

As much as I love EQII and love working on the game, there comes a time when you are ready for a change.  I could easily have been happy working on EQII for many more years, but I decided that I really wanted to take what I've learned throughout the years and apply it to a new project.  As much flexibility as we had with EQII, there are just some things you can't change with a mature game.

The next line in the EQ franchise, currently referred to as EQNext, has been in early development for a little while.  It really made perfect sense for this to be the project I moved to.  I still have a passion for the genre and for the EQ lore and this seems like the perfect place for me to build upon my experiences from EQII.  Two weeks ago I made the move to the new team and have begun settling in nicely.  It's so very different working on a pre-launch game and I absolutely love it.  There are still milestones and deadlines so the pressure is there to perform, but for the first time in 5 years I can dedicate 100% of my time to making a game without dealing with the distractions of maintaining a live game.  Don't get me wrong, those distractions can be fun too, but it makes for a challenge to get other things done when you have to deal with random crashes, feature requests or CS problems to name a few.  :)

I'm definitely going to miss EQII and you'll probably see me lurking around the forums late at night just to keep up with the latest developments.  But at the same time, this new project represents a whole new adventure.  And in many ways, that same excitement that I had when I first joined SOE 5 years ago has returned.  I'm not sure exactly what challenges come next but I'm really looking forward to it!

To my in-game friends from EQII, too many to name, you guys don't know how much of an inspiration you've been to me.  The EQII community has so much passion for the game that many times it was the sole driving force behind my desire to come to work every day.  I'm going to miss the community very much but look forward to the day that we can release a new generation of game on you.  :)

Let's not be strangers!

Tuesday, October 18, 2011

Going Solar!

I've been researching solar electricity systems for a couple of years now but the systems have always been cost-prohibitive, especially if you don't plan on staying in your house for a long time.

Several companies are starting to market a new lease option where you don't own the solar equipment, you just lease it. This is a very attractive plan because you start saving money in the very first month. As long as you have good credit, you can get a lease option with zero money down. The lease payment increases slightly over time, but this should balance with the increase in electricity costs over the years. The other advantage to an equipment lease is that you are never on the hook for maintenance or repairs. The lease may not be good for everyone, but it's a very attractive option for most people and I highly encourage you to look into it if you're thinking about solar.

After researching several options, I decided to go with a company called Sungevity. They have a very attractive lease package and some nice promotions going on right now. They can have an initial solar design to you via email in a matter of days, maybe even hours, by using Bing maps to site-survey your house online. Initially they designed a pretty small system for me. I requested that I wanted to maximize my savings even more so they came back with a larger system. The larger system carried a higher lease payment, but generated more power. For my system proposal the lease payment would be $89 dollars per month. Sungevity guarantees a power output each year, so if the system under-performs they write a check for the difference. Their guarantee for my first year was about 6100 kilowatt-hours. This averages out to about 510 kWh per month. Given that those units are in SDG&E's tier 4 price structure, they cost me about .31 cents per kWh. 510 * .31 = $158 dollars/month savings based on their guarantee. Since I'm sure they don't want to write checks to their customers, I'm assuming that my actual system output will be higher than this and hope for something between 550-600. Of course this will depend on the time of the year as well, so I'm only talking averages.

So, with a $160 dollar savings to my utility bill, minus the $89 dollar lease payment, I'm guaranteed a savings of about 70 dollars per month starting with the very first month. The only downside I can see at this point to signing the lease agreement is that I'm responsible for the lease for 15 years. If I want to sell my house before the lease is up, I have a couple of different options. One is to pay out the rest of my lease but the more attractive option is to sign the lease over the buyer. As long as the buyer qualifies credit-wise, they can assume the lease payment. If the buyer sees the savings that I'm getting every month, it would be a no-brainer for them to take over the lease. The hard work has been done, they can just enjoy the savings every month.

When the lease is up, you can renew it again, buy it out, or let them pick up the equipment. Sungevity includes free monitoring as well as all maintenance. This makes sense given that they guarantee the system output. If a panel breaks, becomes blocked or needs to be cleaned, its in their best interest to do it.

I think I'll be really happy with this system and I'm looking forward to the install. Unfortunately they have about a 14-week lead time because this requires pulling permits and dealing with the power company. I'm pretty impressed with what I've seen so far and will keep my blog updated as to the progress.

If you're interested in solar I'd highly recommend you talk to my salesperson, Jarrett Holt. In addition, Sungevity has a nice referral program. If you use my referral number we'll both receive $500 dollars on top of any other incentives. Be sure to mention my name and referral code of 115952 when you call.

Timeline
10/14/11 - Signed the lease agreement with Sungevity
10/18/11 - Received a call and follow-up email from my project manager, Ian Petrich. We scheduled the on-site survey for 10/24.
10/19/11 - Received FedEx packet with contracts and authorizations to sign.
10/20/11 - Contracts sent back to Sungevity via FedEx
10/24/11 - Asgard Electric arrives for the site survey.  They inspect the attic, roof, internet router and electric system.
11/2/11 - Received follow-up email from Ian letting me know that they are reviewing the results of the site survey and making sure that the initial plan meets local building codes and will work for the roof.
1/12/12 - Received an email sometime back in December that my new project manager is Jan York.  Spoke with her today about my system and they have to make some changes based on local codes.  I also informed them that the new roof is going on the house today which takes care of any issues they had with the roof quality.  The will be finalizing the system with the new information and hopefully getting back to me soon.  It sounds like because of fire codes in this area they have to keep a 36" walk space around all panels which gives them less roof to work with.  They asked that I relocate my satellite dish if possible to increase my system size.
3/20/12 - As of a month or so ago I was assigned a new project manager.  Today I contacted them about my install date since it had been awhile.  My PM called me back and said they were able to expedite the install to April.  They also were able to upgrade my panels from Schuco 185's to 190's which produce a little more power (5 watts per panel).
3/21/12 - The electrical plan was submitted to SDG&E and approved within a couple of hours.  SDG&E states that their next step is to inspect the system after it is installed.  Hopefully we'll get an install date from Asgard Electric soon.
4/16/12 - Tony at Sungevity called and said they were trying to get Asgard to do installation this week and would get back with me later.
4/18/12 - Tony confirmed that Asgard would start installation on 4/19 completing installation on 4/20 and scheduling inspection for 4/23.
4/19/12 - Asgard arrived today and installed the solar unit on the side of the house and began attaching the panel anchors to the roof.  http://gspence.smugmug.com/Projects/Solar-Install/
4/20/12 - Asgard spent today finishing most of the installation.  At the end of the day there were still a few things to be done.
4/23 - 4/27 - Asgard made several trips out to tie up loose ends and complete the installation.  The city inspector also came and passed inspection on the entire system.
4/30/12 - SDG&E inspected the system and said we were now able to being using it.  They switched us to a NET metering account which is a special account that you only need to pay yearly for your energy usage.  With SDG&E, if you produce more than you used in a month, you receive energy credits you can use during another month where you didn't produce as much.  SDG&E will not currently pay you for overages.
4/31/12 - Today was the first day of operation.  Unfortunately it was a cloudy day, but still produced 15 kWh of clean energy!
5/1/12 - Produced 19 kWh
5/2/12 - Produced 20 kWh

Monday, November 29, 2010

Christmas time is here!

Ever since I saw someone post a video of Christmas lights animated to music, I thought "I want to do that!" That was probably 5 years ago and every Christmas I think about it but never take the plunge. My friends know that I'm the type of person to go overboard in just about everything I set my mind to doing. So having an excuse to load up my house with as many Christmas lights as I can, and sequence them all to popular holiday songs is right up my alley.

This summer I decided to order the hardware to start off with a modest 32 channel light display. I also put in a large order of bulk wire and Christmas lights. My first order of lights looked something like this:

50 strands of white lights
50 strands of green lights
50 strands of red lights
250ft bulk spool of C7 wire with empty sockets
300 roof clips
150 white C7 bulbs
100 green C7 bulbs
100 red C7 bulbs
1000 ft bulk SPT-2 lamp cord
36 male/female ZIP plugs

I came to find out pretty quickly that this would be nowhere near the number of lights that I would need so I started hitting Home Depot, Lowes and Wal-Mart as soon as their lights were out. In the case of some stores, this was weeks before Thanksgiving and I got some really strange looks and odd comments as I checked out with Christmas lights so early in the year. I could only think to myself, "hey, if you're going to put them out for sale, don't you expect people to buy them?"

Over the next few weeks I purchased 10's of thousands of more lights. The bulk of the lights make up the four large trees in my front yard. I also created 8 mini-trees that each have about 300 lights on them. The roof is lined with 3 channels of C7 lights as well as about 2000 icicle lights. The star of the display (literally) is a 4' x 8' star that I created out of plywood and 2x4's. It holds about 200 C9 lights split up into 3 different channels. The design of this star was inspired by someone else on the internet and it was so amazing that I decided to create one just like it. The star now sits on the roof of my house weighted down by 150 lbs of sandbags.

I really procrastinated on sequencing the music and didn't start on it till a few days before my display was set to go live. Fortunately I had some practice sequencing lights for a friend's Halloween party so I was already familiar with the software. I was able to get two songs done before Thanksgiving; Christmas Canon Rock and Wizards in Winter, both by Trans-Siberian Orchestra. Friday night, the day after Thanksgiving, I powered up the light show and everything worked flawlessly. It was really cool seeing all that hard work pay off and the comments I've gotten from friends and neighbors have been really positive.

I've been working on more songs a little each night and now have two more added to the show; Let It Snow by Michael Buble and the theme from Christmas Vacation. I hope to add 2-4 more songs in the next couple of weeks.

So, if you find yourself in the San Diego area, drive by 452 Highland Place in Escondido and check out the show! You can tune your car radio to 93.5 to hear the music or step outside and hear it from the sidewalk. If you can't make it by, pull up my channel on YouTube and take a look at some of the videos I've posted. More videos will come as I add new songs.

My YouTube Channel
(Be sure to subscribe if you want notifications when more songs are posted)

Thanks for reading and Happy Holidays!

Wednesday, September 29, 2010

Good stuff coming in GU58.

Game Update 58 for EQII will bring a lot of nice additions to the game. When the update notes are posted, you'll see just how many things we've fixed and added over the past few months.

In the meantime I'll give you a little preview of the things I've been working on.

NPC Casting Bars

Casting bars have now been added to the target and implied target windows that will allow you to see exactly what spells that NPC's are casting. You've always had this information in one way or another normally by viewing particle effects or reading the spell text that appears in your chat window. This would allow tools like ACT to parse the data and give you the information, but for most people that don't use these tools, it was more difficult to understand what the mobs were doing. Hopefully with this new feature you'll be able to better time your stuns, stifles and interrupts.

Mount Appearance and Inventory

We've added a Mount tab to your character window, right behind the Appearance Tab. You can now store up to 50 mounts in this special inventory window and get them out of your normal inventory. In addition, you'll be able to define which mount is your "Equipped" mount and which mount is your "Appearance" mount. Following in the footsteps of appearance gear, the Equipped mount will provide the buff and stats, but the appearance will be overridden by the Appearance mount. We've also create a new "Summon Mount" ability that appears in your knowledge window as well as the bottom of the mount tab. Clicking this button will automatically summon/unsummon the equipped mount combination.

Currency Window

Behind the mount tab on the character window, we've also added a new Currency tab. Currency tokens and items will now be stored in this window freeing up even more inventory space. You can drag tokens to and from this area, so tokens can still be tradable or placed in your bank. When purchasing items from vendors that require tokens, the vendor will consume them directly from the currency window. Also, when acquiring new tokens, they will be deposited automatically into this window.

We also have a huge list of fixes and features that came from Fan Faire so I hope everyone really likes this update. We have a bit longer cycle between GU58 and GU59 since 59 is the launch of Velious but we are planning for a few things to be released in hotfixes between now and then. We also have 3 holiday events happening between 58 and Velious so hopefully there will be plenty to keep you busy!

We've also just announced that character slots will be going on sale through the in-game marketplace at a price of 10 dollars (1000sc) per slot. When you buy this slot, it'll be permanently added to that account and won't increase your monthly fee.

Thanks for reading and thanks again for being a Norrathian citizen! :)

Wednesday, August 04, 2010

Mighty Mugg

This year our community department came up with a really cool idea to have SOE employees create custom Mighty Muggs to be auctioned off at Fan Faire. I've always loved painting miniatures and creating custom terrain for tabletop wargames so this sounded like something I'd really enjoy doing. I can't wait to see what some of the other employees have done. Domino also posted a blog of her excellent Mugg painting.

I decided the theme for my Mugg would be my GM character, Rothgar, who is always parading about as a Guardian in full plate. I started with a blank, white, Mugg as seen below. (Ok, i cheated and turned his head around because I forgot to take a pic before I started).



The next step was to use some Sculpey to form the platemail helm. I don't really have a lot of experience sculpting, so I had to experiment with creating shapes and carving out grooves. With a little practice, the Sculpey wasn't too hard to work with. You just have to soften it up first. After forming the front part of the helm, I cut out the eye slit, carved grooves for the plates and bent out the chin piece.


Next I rolled out a really thin tube of Sculpey to use as the edging around the helm plates. It was hard to make this part stick, but with a lot of smearing and scraping I was able to get it to adhere to the main piece. I really hoped that everything would stick better after baking.


Next I created another long flat piece to form up the back of the helm. This had to have a slight bend at the bottom and form as seamless of a line as possible at the top. After attaching the back piece, I created two circles to represent the hinge piece and then added the spikey bits above them. In the actual picture of the helm, the spike bits stuck out from the head but they would have been too brittle so I sculpted them as part of the helm.


Next was to create the top of the helm. I rolled a piece of clay, attached to the head and then squared it off. Making the top detail was kind of a pain. I rolled out a flat piece and tried to taper it as much as possible. I traced the curve of the head and cut out the design as close as I could with a knife. Once I got it placed I did some more carving to taper the tops and then carved the indents to separate the pieces more.

 
At some point after this I got lazy and stopped taking pictures. I continued sculpting a belt and the flaps (not sure what they are called) for the front and back of the plate greaves. Since the body of the Mugg was already pretty stocky, I decided to paint the detail on the chest and arms rather than build it up any thicker with clay.

I sculpted a sword by starting with the hilt. I rolled out another tube and tapered it towards one end. I kept checking the fit and tapering more until I got it to slide easily into his hand. I cut the guard and blade from more sheets of Sculpey. I used an edging tool to press grooves into the hilt and then carves the blade to the proper shape and put an edge on it. The guard is kind of simple. If I had more time I probably would have done something fancier.

The last bit of sculpting was to create a shield. I looked at a few different designs in the game and went with a shape that I have always liked. Making these shapes uniform on both sides is always a pain so the first thing I did was to draw half of the shield on a piece of paper. I then cut it out and used it as a template to make sure that both sides of the shield were identical when I cut them out of the clay. I carved a few grooves and the shield was done.

Now that everything was carved, I had to figure out how I was going to bake the clay so that it would harden. The instruction said to bake it at 275 but I was sure I'd melt the mug. Since there was no way I was taking the clay off, I had to determine a safe temperature to bake it at. Searching around on the web only found me a link where someone baked their Mugg too hot and melted the head. So I decided to try 240F and checked it often. The Mugg got soft but held its shape. I baked it for about 20 minutes, let it cool and then repeated the process several times. The clay ended up hardening nicely and my Mugg never lost it's shape.

Now for paint. I primed the white Mugg with Games Workship black primer. I prefer black for most things I paint because its easier to build up and leave cracks and crevices dark. Its just harder to make bright colors on a dark primer, but its works pretty well for metals. I dry-brushed the entire model with silver giving it a nice finish. Using black I darkened up the recesses. Using pure silver I painted the highlights.

The next task was to paint the detailed lines on the armor. I decided to start with the light blue and then outline the blue lines on both sides with black lines. Wow, such a pain. Again, I had to try to make sure that everything looked equal on the left and right sides of the Mugg. Thinking back, I probably should have drawn the lines first with a pencil but I love doing things the hard way.

After finishing all the paint, I went out to the garage to spray him with a clear-coat. I was hoping I had a clear gloss but all I could find was a matte finish. With the Mugg balanced on a piece of wood, being held like a waiter carrying a tray of food by my left hand, I sprayed the model with my right hand. Unfortunately I never held a job as a waiter and at some point my hand got too shakey and the Mugg took a dive. I stuck my foot out to break the fall but he still landed head-first on the concrete floor. I could have cried. Fortunately there wasn't much damage. A piece of his head-spike broke off but I was able to find it and glue it back on. A little touch-up paint and he was almost as good as new.

My next attempt to clear-coat him was done with him laying securely on his back. After some dry time I flipped him over and did the other side.

Whew... what a long project! I had a blast doing it though and I hope whoever wins him will be happy with the work.

Tuesday, July 20, 2010

SOE Fan Faire 2010 & iPhone App

Fan Faire is just around the corner. Last year I created an iPhone web app to assist people in finding panels and other Fan Faire related information. The web app used Safari extensions to give the look and feel of an actual iPhone app, but since it was served from the web, it required an internet connection.

This year since I had more time to think about it I decided to create an actual iPhone app this time. The app was submitted last week and is now available for download from the app store, just search for "SOE Fan Faire".

A new version will be going up soon with updated maps and hi-res graphics for the iPhone 4's retina display. I'd be interested to hear your feedback so let me know what you think. However, I'm not interested in getting into a debate of iPhone vs Android vs Some-Other-Smartphone.

Tuesday, April 20, 2010

Speaking at the 3D Gaming Summit on April 21st

I have the pleasure of speaking at the 3D Gaming Summit in Universal City this week. I'll be on a panel Wednesday afternoon entitled "Creating State of the Art 3D Games: Lessons from the Field".

They've asked me to give a demo of EverQuest II which now supports NVIDIA's 3D Vision technology. EQII looks truly amazing in 3D. I'll also be talking about our experiences working with their API and covering some of the typical issues other game teams might encounter when working with this technology.

I'm excited to be speaking in front of a big crowd but I'm also a little nervous as well. I'll be demoing the game on live servers using a machine that I won't get to check out until Tuesday night. So, there's always the possibility of that random client crash in the middle of the demo, and that would be quite embarrassing. So cross your fingers for me and hope that everything goes smoothly. :)

Tuesday, February 16, 2010

Sentinel's Fate is Live!

Our QA department passed the servers and everything has been unlocked. Servers are up and people are online. So far everything looks great. I'm sure there will be minor issues throughout the day, so I'll be keeping my eyes and ears open for them.

Enjoy the expansion and let me know what you think!

Launch Update

It's almost 5am on the morning of launch. A few others from the game team have arrived.

The DBA's have completed the database schema changes and our Operations department should have published the expansion code to all of the servers by now. As soon as our QA department arrives the servers will be brought up for a smoke-test.

We just finished making some last-minute optimizations to our world server. This server is responsible for handling communication between all the zones as well as caching worldwide data such as vendors, guild bank items, etc. The issue on Nagafen yesterday seemed to stem from the worldserver hitting the process memory limit while loading all of this data.

So tonight we went over the code and found some places where we could drastically reduce memory during loading. We've made those changes, built the new server and copied it manually to Nagafen's host machine. Our initial test of bringing Nagafen up looks good and it didn't crash this time so everything appears to be working. We still need to have QA confirm that everything is working as expect.

In general, we would never make changes in this situation without ample time to test, but the three of us are pretty confident that the changes are low-impact and will allow us to get Nagafen back online. Worst-case scenario we just use the old version and keep the broker offline a little longer.

We don't plan to roll this version out to other servers though until it has been properly tested by QA.

3, 2, 1, Launch!

It's 2am and I'm now at work. Curretly there are only a few of us from the game team here; myself, Nandy Szots, John Rohrssen, Nick Parkinson and Paul Molina.

However, our operations department and database admins are also here and have already started preparing the servers for the expansion.

There isn't as much to do until the servers have been updated other than to start looking into a problem that Nagafen had last night.

Monday, February 15, 2010

The Eve of Sentinel's Fate

It's about 8:30pm on the night before we launch Sentinel's Fate, the 6th expansion for EverQuest II. This is also my first launch as Lead Programmer so I'm pretty excited albeit a little nervous.

This expansion hasn't been without it's challenges. Most of the team has been on a crunch schedule for several months and it still feels pretty frantic. It's kind of like when you're leaving the house to go on vacation and you just know you're forgetting something.

That's enough for now though. I'm going to try to get a few hours of sleep before heading into the office. Check back later for more blog posts throughout this launch event!

See you in a few hours.

Thursday, November 05, 2009

New Quest Helper coming soon.

Yesterday we branched our code for GU54 which means we're safe to start checking things into our code repository for GU55 (Sentinel's Fate).

I've been working on the new Quest Helper window for awhile and today I was able to finally get this work checked in. I'm pretty happy with the results so far. We've wanted to make this change for a long time but just hadn't found the time to do it. Since we're planning on a big marketing push in February, we're trying to get in as many features as we can that will improve the experience for new players.

One of the biggest things that bothered me when making a new character was the fact that we could only track 1 quest at a time. This just doesn't work in the current environment where it's typical to grab 3-5 quests from a "quest hub" and go do them all at once.

I'm sure this window will be familiar looking if you've played other MMO's that have a quest tracker for multiple quests. There just aren't too many ways to innovate in this area. The goal is to present a list of to-do items in a concise and organized manner. I think this window delivers on that and I hope everyone will like it when it comes out in February.

Saturday, October 24, 2009

PvP Merge Recap

It's been awhile since I've had a chance to blog and I really need to make a concerted effort to do it more frequently. It's certainly not for lack of things to blog about, but more about making the time to do it. I think if I stop worrying about writing long blog posts and try to write smaller updates, it'll be easier for me to keep up with it.

We've been preparing to merge the PvP server Venekor into Nagafen for several months now. We announced it around the time of Fan Faire and set a date for some time in September. As we approached the date we started experiencing performance problems on Nagafen's database that gave us some real concern about whether or not it would be able to handle the additional load from Venekor. Nagafen's performance has always been a little sluggish due to the higher population but a few weeks before the merge, the problem got much worse.

We had several meetings with our DBA's to discuss methods for improving the performance. Most of the options were band-aids or had no guarantee that they would solve the problem. Much to our surprise the DBA's were able to offer a complete upgrade to the database hardware and software. This was something we had been wanting to do for awhile but hadn't had the opportunity due to conflicting projects for the DBA Team. While this was certainly the most desirable option, there just wasn't enough time to do it before the merge that was only 2 weeks away. Although we knew that the decision would be met with some disappointment, we decided to delay the merge by another 4 weeks to give us time to upgrade the database hardware.

We scheduled the migration of the Nagafen data to the new database a week before the merge was to happen. The migration day came, the servers were brought down and the data was moved to the new database. Everything appeared to be perfect even after we brought up the servers on the new hardware. Unfortunately, there was a small subset of characters who were not able to log in. After some research we found that the migration software had a problem moving certain large BLOB fields. At this point we had to rollback to the old database and come up with a new plan. After more meetings we decide to try the migration again but resort to a different method of moving the data that was slower but more reliable. Several days later we tried again and everything worked great.

The DB migration happened on a Tuesday. Ideally we would have waited until the next Tuesday to perform the merge, but this would mean delaying it another whole week and we just didn't want to do that. So we decided to kick off the merge process Wednesday night at midnight. We had a 28 hour maintenance window to complete the merge.

On the evening of the merge we kicked off the script to export the data from Venekor. The script was running much slower than during our tests and looked as if it would take 26 hours just to export the data which was only half of the process. At this point we didn't have a lot of options so we let it go. By 7am Thursday morning the script had been running for 7 hours when it decided to crash with a segmentation fault. You can imagine how we felt knowing that the script was already taking twice as long as it should have been and on top of that we just lost 7 hours. We decided to run the script from another machine to rule out hardware problems and started it up again. That morning we discussed worst-case scenarios and pretty much decided that if the script crashed again we would have no other choice but to abort and troubleshoot the problem. However, not only did the script continue to run, it actually started running faster. The export script completed after about 15 hours. The next step of the process was to bring down Nagafen for a final backup and then start the import of the data.

The import went a good deal faster since it was running against the newer database hardware. About 12 hours later the import script finished and all of the Venekor data was now in Nagafen's database. There were still many things to do and fortunately the timing worked out so that the import finished during the day and not in the middle of the night. This allowed us to tackle the remaining tasks from the office during normal work hours. We continued to execute scripts that copied shared banks, guild halls and more. These scripts had to resolve name conflicts and keep up with ID changes so we could notify the chat and mail servers of changing player data. Chat servers were brought down to update their data which unfortunately had to impact all game servers. (Sorry about that)

As soon as all of the scripts were done we were ready to bring up Nagafen and have QA run through the standard sanity checks to make sure everything looks OK. Our smoketest takes several people an hour or two to complete and covers most of the gaming functionality. Unfortunately its just not possible to test every detail and every code-path. The smoketest passed and the server was unlocked.

For the most part, everything looks good. We haven't seen any problems with character data other than some issues with missing shared bank coin. We also had an issue with guild hall supply depot contents not transferring over correctly. These can be corrected with some additional time and overall I'd say the merge was a success. The server has only been up for about 5-6 hours at the time of this post, so I hope we don't discover any other issues. *Keeps fingers crossed*

Some of us will be busy fixing up these final issues over the next couple of days, but overall I'm very happy with how well things turned out. Everyone who had a part in the merge and DB migration did an excellent job and I'm proud to be working with a group of such intelligent and hard-working people.

We understand that the extended downtime was an inconvenience to those of you who played on Venekor and Nagafen. We apologize for that but hope this process will bring you much more enjoyment in the long run.

Thanks!

Saturday, June 13, 2009

Fan Faire iPhone Web App


I've been working on an iPhone Web App for our upcoming SOE Fan Faire in Las Vegas. The goal of the app is to give iPhone users an easy-to-access reference for all information they might need while attending Fan Faire.

The app will have the following information:
- Full schedule of all events and panels browsable by game.
- A map of Bally's convention center
- Twitter feed from our @soefanfaire twitter account for up-to-the minute news
- Extensive list of restaurants around Bally's and Paris hotels.
- Travel information for getting around Vegas

I threw together the basics of the web app within a couple of evenings of work. Now I'm working on the polish features that make it really useful.

Being able to filter the schedule by game is a cool feature and will help attendees find panels without scrolling through the entire list of events.

Originally the list of restaurants was going to be very basic but my wife encouraged me to go the extra step and provide more info about the restaurants. So each restaurant now has a detail page that shows their location, hours, type of cuisine, cost and description. Clicking the phone number will initiate dialing on your iPhone. There's even a button to take you to the restaurants website if you're interested in browsing their menu. (Thanks Tiffany for typing all that data in!)

The news section fetches the RSS feed from twitter and displays the last several entries in a list. If the twitter entry contains a URL, clicking the row will open the browser to the web page.

There are still some things I need to finish before I can call it done but its coming together nicely. If you'd like to see it while its in development, you can check it out at http://www.gregsplace.com/fanfaire. Keep in mind it will only work in the Safari browser because it makes heavy use of their iPhone extensions.

Let me know what you think!