Dev Notes: Week of December 29, 2014

[RhythmNBruise]

Hey, guys! Sorry these are late... I was waiting on our special guest columnist. This week, in lieu of proper dev notes, I'm bringing you a letter from our QA lead here on MW!



Hi,

I am the QA lead for Modern War, and we over here at GREE wanted to let you guys in on a little bit of the process and struggles that we go through to better inform you what it is we actually do.

For starters, my team is much smaller than our user base, but we still have to to run through the new feature flows and every single event. It’s a lot of work, and a large portion of my job is risk assessment. There are thousands of different cases to any feature, and we have to identify the highest risk areas and test them. If you guys think that an area of the game is buggy, or a new feature is buggy, you can’t even imagine how buggy it is when we first get our hands on it :)

For a deeper look: when it comes to events we have test plan templates that we make copies of to run through the event flows, checking mostly for text errors, missing art, correct item stats/bonuses, etc… With new clients, we actually have to create a test plan for the feature, mainly focused on checking boundary cases and other normal functionality/design flows. After that, we go through bug turnaround waiting for engineers to unblock portions of the feature, fix bugs, or change things based on our suggestions. Depending on time we do get in some edge cases (often we don’t get to run them because they are more time consuming), and pretty much all of our corner cases are left to be tested by you fine ladies and gentlemen. Unfortunately we will never have the bandwidth (and this is every company I’ve worked at, not just GREE) to test all concurrency issues. Me and my team just cannot put nearly as much stress on our servers as the you guys can. We try to supplement this by running betas, but even that doesn’t always bring all the issues to light.

I’d also like to mention that, with account transfers as an example, QA put in 30 man hours a person over a single weekend; and that wasn’t nearly all of the testing we did. We did our best to make sure as few of you would have account loss problems as possible. However, we all knew that on release there would be struggles, edge and corner cases, etc.. But the CS and Server teams were as prepared as possible to get you guys back any lost accounts as quickly as we could manage.

I hope this helped shed some light on the hard work we do to try to make the game as high quality as we can.

Thanks,

I hope to see you guys on the battlefield!



Do you think you have what it takes it join the QA Team?

Check out the QA Game Tester Job requirements (http://product.gree.net/us/en/careers/view?&jvi=oXfd0fw9,job)!

[Genen]

Hey, guys! Sorry these are late... I was waiting on our special guest columnist. This week, in lieu of proper dev notes, I'm bringing you a letter from our QA lead here on MW!



Hi,

I am the QA lead for Modern War, and we over here at GREE wanted to let you guys in on a little bit of the process and struggles that we go through to better inform you what it is we actually do.

For starters, my team is much smaller than our user base, but we still have to to run through the new feature flows and every single event. It’s a lot of work, and a large portion of my job is risk assessment. There are thousands of different cases to any feature, and we have to identify the highest risk areas and test them. If you guys think that an area of the game is buggy, or a new feature is buggy, you can’t even imagine how buggy it is when we first get our hands on it :)

For a deeper look: when it comes to events we have test plan templates that we make copies of to run through the event flows, checking mostly for text errors, missing art, correct item stats/bonuses, etc… With new clients, we actually have to create a test plan for the feature, mainly focused on checking boundary cases and other normal functionality/design flows. After that, we go through bug turnaround waiting for engineers to unblock portions of the feature, fix bugs, or change things based on our suggestions. Depending on time we do get in some edge cases (often we don’t get to run them because they are more time consuming), and pretty much all of our corner cases are left to be tested by you fine ladies and gentlemen. Unfortunately we will never have the bandwidth (and this is every company I’ve worked at, not just GREE) to test all concurrency issues. Me and my team just cannot put nearly as much stress on our servers as the you guys can. We try to supplement this by running betas, but even that doesn’t always bring all the issues to light.

I’d also like to mention that, with account transfers as an example, QA put in 30 man hours a person over a single weekend; and that wasn’t nearly all of the testing we did. We did our best to make sure as few of you would have account loss problems as possible. However, we all knew that on release there would be struggles, edge and corner cases, etc.. But the CS and Server teams were as prepared as possible to get you guys back any lost accounts as quickly as we could manage.

I hope this helped shed some light on the hard work we do to try to make the game as high quality as we can.

Thanks,

I hope to see you guys on the battlefield!



Do you think you have what it takes it join the QA Team?

Check out the QA Game Tester Job requirements (http://product.gree.net/us/en/careers/view?&jvi=oXfd0fw9,job)!
Can we have health regen prizes back as WD and ltq awards? Only one that comes now is 1% health regen for 1st place in division during WD. Also health temp prizes are drastically reduced to 5% for entire cycle. Thx.

[Genen]

Also devs mentioned before that it will be minimum scoring requirements for each faction member in order to receive prizes during WD. Will this be instituted for upcoming WD? If yes could you tell us please what this requirement will be. This information will help faction leaders when they recruit new members.

[andy2]

Sounded like a lot of excuses to make it sound OK for putting out an oft broken product. Some of them legit, many not. Either way, the biggest issue that you have is that when problems are found we get very little to no communication or support from support staff to help us through the issues or recover from them. Here is a list of many of the current bugs - how about actually confirming you are now aware of them and giving us a status.

http://forums.gree.net/showthread.php?103578-Comprehensive-List-of-Bugs-Issues-in-MW

[Annihilator2]

You have the same problems as all development groups:
- developers are coding without thinking; unit testing is not performed or performed formally because of time limitation;
- QA team is small and tests only declared changes including new features, bug fixes, improvements but does not return to "not changed" functionality. Not all scenarios are tested because of staff and time limitation;
- CS team is also small and is not enough professional.

So your problem is business and general management that kills the game.

It is common today for all development teams in software and hardware in all areas, not only gaming.

[MW Flake]

I agree Andy. They've just confirmed what we knew all along: the customers are the testers. If those guys find the biggest, baddest bugs and glitches, and we still get massive problems on our end, then what does that say of the skills of the coders? I don't play a lot of app games, but the junk that gets released to end users, sorry end testers, is far worse here than in any other games I've played. CS is still not getting accounts back, 3 weeks after your hard work gave us yet another broken release so obviously they were not all that prepared. My question is, after hearing at how hard devs work to ensure a quality product, how hard QA works to ensure a quality product, and how hard CS works to handle the problems with said quality product, how hard is the person in charge of all of you working since we never actually get a quality product?

I know my comments are harsh. I don't say them to be mean, I am just a very direct individually. Yes my words are critical, but unfortunately it is the reality we, as your customers, see. Your customers are not happy people. I play a competitors game, and in nearly a year I have yet to encounter a single glitch. Not one, yet every single event, every single day of Modern War is filled with them. Some minor, many not. And it's only getting worse as time goes on, never better.

Many forum old timers complain about all the "whiners" that currently frequent the forum, and long for the old days when it was players helping players. Well guess what? All those "whiners" do too! But bug after bug after glitch after bug have made it all but impossible as we deal with bugs more than we are able to actually play anymore. Let the higher ups see this. This game isn't sinking because it's a bad or now boring game, it's in large part because nothing ever works anymore. The other large part of that equation about the disconnect between what players want and what you give us is a whole separate discussion for another time.

[Hobbs]

Yet another CLOSED TICKET !! Regarding the missing Equipment from faction and individual "grinchy" prizes?
If you are gonna introduce this Awful Equipment idea at least have it work right,as equipment was awarded in this fltq it was actually subtracting from equip stats so was getting weaker as I progressed and know of a lot more with same issue.
All tickets closed with a BS run down of how new battle system works,this is one example why this game is imploding the only thing that you can count on is error after error and no decent help when they occur.

[Bluehavana2]

I've worked a couple of decades in support and development, so I understand the challenges, but at some point a release needs stability before continuing development. How about freezing existing functionality and get things fixed before going back into development of new features???

[andy2]

I dont work in the gaming industry, but the technology industry specifically as a developer on a QA team and I can say for sure that if we had this many customer complaints we would 1) be out of business and 2) if we somehow weren't all of the teams from QA to development would be held accountable... and if we tried giving these types of excuses to customers would be fired in a heartbeat.

[DieselPowered]

So let me get this right, you dont have enough manpower and time to test the game/update properly before you release it to us. So who sets the deadline for release of the product to the end users (testers in your eyes)? Many of us pay you for this product. I dont pay Ford to test the truck that they sold me. I dont pay Matco/Snap-on to test their tools for them. I dont pay Mcdonalds to test their food for them. Why are we paying to test a game for Gree? Id say, at this point, that its time for a recall of the product known as Modern War 5.0. (even free players are not recommending this game to their friends) Have any of the Gree employees/management seen any of the many groupme, line, palringo, etc channels that used to buzz with the activity of the events that are now basically all recruitment channels from factions that have lost many longstanding members and are having trouble filling their rosters?

Why cant the deadlines be pushed back until the product is finished? From the sounds of it, many of us could have waited many, many, many more years before the release of 5.0 and this new crap equipment. Instead of pushing out new unproperly tested updates, how about fixing whats wrong (all of it) and properly testing it before working on the next new shiny baubles, bugs, glitches, and "display issues" that will erase the years of work and dedication of your user base (those putting the money in the bank for your paychecks)

[East Coast Bias]

This is like the n'th time I end up scratching my head wondering why you've decided to be so blatantly honest. This is exactly what we expected, but never really wanted you to admit that you've been knowingly pushing out a product that is full of glitches, errors, and whatever else. And we're not stupid, we know it's not your fault, the mods fault and the list goes on. Rather than having you guys play the role of sacrificial lamb, I'd rather have Mr. Tanaka or someone from corporate who makes actual $$ decisions do a Q&A session. I'm sure you guys are trying your best despite the lack of tools, staff, etc. But someone who's looking at the P&L is pushing this **** out the door. I'd appreciate it if the mods can bring this up and arrange such a forum. Forget the devs, what we want is a session with corporate.

[bam bam.]

Thanks for the letter. We do think its buggy, especially those of us still locked Out. If you can't keep up...slow down!. Please slow down, as 85% of our community has asked for a couple days of a swing (Ill bump the pole so you can see it on the feed again) there really is no sense of urgency. Please do it once and do it right. Listen to us. Thanks for the communication, keep it up, it is really appreciated.

[jps01]

Hi Guys,
Still unable to equip soldier with new equipment. Just get server error. Any news on this problem? Reported ticket last week.

[Pidgeot]

We would also like more health and energy temp bonuses. Now we get only one a cycle. These dont help us. Would also be nice to see some health and energy regen boost at the lower levels. The only one I have is the +1 from Nuclear Soldier.

[redvenge99]

I as a long time player, think this kind of thread and communication is very appreciated and welcome. Its a giant step forward, it helps to keep the community as a whole informed and involed. Yes there are issues in the game, even xbox and playstation deliver gsmes with known or unknown issues. You cant please everyone all the time and honestly you can only test certain things for a certain environment, which is hardly the real world.. every company in the world treats there customers as testers. Why do you think everything comes with a warranty? The main thing that has and does have ppl in an uproar has been the total lack of communication from the staff and developers, this has begun to change, thanks for the insight and honesty. Hopefully things continue to get better. Keep it up gree.

[FLGuy]

If we update to 5.0 and lose our player profiles, and get 0 response on our tickets, 0 response from moderator PM's..... Who cares about future discussions, features and bells and whistles!

[Sparkle]

[QUOTE=RhythmNBruise;1444656]



Hi,

I am the QA lead for Modern War, and we over here at GREE wanted to let you guys in on a little bit of the process and struggles that we go through to better inform you what it is we actually do...
If you guys think that an area of the game is buggy, or a new feature is buggy, you can’t even imagine how buggy it is when we first get our hands on it :)

{You just minimized our concerns}

... Depending on time we do get in some edge cases (often we don’t get to run them because they are more time consuming)...


{You have all the time in the world to leave the basic game alone while you work out the problems with new ideas. Just hold your horses until you get it right.}

... and pretty much all of our corner cases are left to be tested by you fine ladies and gentlemen. Unfortunately we will never have the bandwidth (and this is every company I’ve worked at, not just GREE) to test all concurrency issues. Me and my team just cannot put nearly as much stress on our servers as the you guys can...


{I have a great idea! I work in Healthcare. How about I try out some new stuff on you and see what happens? And I will charge you a whole month's worth of rent to do it!}

... We try to supplement this by running betas, but even that doesn’t always bring all the issues to light...

{What does it take to get to be a beta tester anyway? I applied twice and didn't even get a sniff. Meanwhile, my friends are getting a bunch of cool stuff. I can't even find the application site anymore. Where is it please?}


... I’d also like to mention that, with account transfers as an example, QA put in 30 man hours a person over a single weekend...


{Yup... long work hours sound impressive but guess what... I worked 12 hour shifts in hospitals 3-4 days at a time. What it does is fry your body and mind then the attitude goes down the toilet. Maybe work less at a clip and you will produce better.}

... and that wasn’t nearly all of the testing we did. We did our best to make sure as few of you would have account loss problems as possible. However, we all knew that on release there would be struggles, edge and corner cases, etc.. But the CS and Server teams were as prepared as possible to get you guys back any lost accounts as quickly as we could manage...


{Surely you realize that weeks to months of lost time in the game sets us back seriously behind the competition. It's one thing if we fail to be active players or have personal things to deal with. Knowing that you guys know that this will happen from you deliberately putting out a flawed product is an infuriating injustice!}


... I hope this helped shed some light on the hard work we do to try to make the game as high quality as we can.

Thanks,

I hope to see you guys on the battlefield!

[SUPpose So]

Hey, guys! Sorry these are late... I was waiting on our special guest columnist. This week, in lieu of proper dev notes, I'm bringing you a letter from our QA lead here on MW!



Hi,

I am the QA lead for Modern War, and we over here at GREE wanted to let you guys in on a little bit of the process and struggles that we go through to better inform you what it is we actually do.

For starters, my team is much smaller than our user base, but we still have to to run through the new feature flows and every single event. It’s a lot of work, and a large portion of my job is risk assessment. There are thousands of different cases to any feature, and we have to identify the highest risk areas and test them. If you guys think that an area of the game is buggy, or a new feature is buggy, you can’t even imagine how buggy it is when we first get our hands on it :)

For a deeper look: when it comes to events we have test plan templates that we make copies of to run through the event flows, checking mostly for text errors, missing art, correct item stats/bonuses, etc… With new clients, we actually have to create a test plan for the feature, mainly focused on checking boundary cases and other normal functionality/design flows. After that, we go through bug turnaround waiting for engineers to unblock portions of the feature, fix bugs, or change things based on our suggestions. Depending on time we do get in some edge cases (often we don’t get to run them because they are more time consuming), and pretty much all of our corner cases are left to be tested by you fine ladies and gentlemen. Unfortunately we will never have the bandwidth (and this is every company I’ve worked at, not just GREE) to test all concurrency issues. Me and my team just cannot put nearly as much stress on our servers as the you guys can. We try to supplement this by running betas, but even that doesn’t always bring all the issues to light.

I’d also like to mention that, with account transfers as an example, QA put in 30 man hours a person over a single weekend; and that wasn’t nearly all of the testing we did. We did our best to make sure as few of you would have account loss problems as possible. However, we all knew that on release there would be struggles, edge and corner cases, etc.. But the CS and Server teams were as prepared as possible to get you guys back any lost accounts as quickly as we could manage.

I hope this helped shed some light on the hard work we do to try to make the game as high quality as we can.

Thanks,

I hope to see you guys on the battlefield!



Do you think you have what it takes it join the QA Team?

Check out the QA Game Tester Job requirements (http://product.gree.net/us/en/careers/view?&jvi=oXfd0fw9,job)!


If you really are the lead of the QA team, why are you not bashing down the door of the CEO and demanding that you have proper programmers who can constructivly appraise what they are turning out. The whole system is a joke, and this inturn casts a very big shadow over your abilities.
I hope you have the gonades to knock on that big door in Japan and demand some qualified experienced progammers.

Read your post above. It is flawed and buggy as excuses go... Happy new year :) and hope we can move forward with a better game structure.