March 7, 2013
I was checking the blog of Fantasy Interactive and was caught by the article about their “Attack” (see video at the bottom of this post). “Attack” is short for “Attack of the Creative Process”, FI’s sessions about their projects. These sessions are now popular among internet trend followers.
In the article (http://blog.f-i.com/so-youre-interested-in-attack/) there is a video of one of the presentations of the “Attack” by FI’s Irene Pereyra and Anton Repponen (@reppnonen).
As a quality focussed producer I was interested in what Irene told about the QA process at Fi.
At 01:07:06 someone in the public asks Irene and Anton what their QA process looks like. Irene told that at first she, Anton and the developer did the testing, but now they have QA people who are testing if the things work the way it should.
This is a big step in internet. I know that a couple of agencies have their QA engineers and I think that is great. I hope more leading agencies will hire QA engineers to assure the quality of their work or the work they receive from third parties. I can assure you that QA will be worth the investment.
The most interesting to me was what Irene answered on the question asked at 01:11:20. I couldn’t hear the question well, but Irene answered with “I hate user testing”. She tells that you can manipulate the result of user testing by asking the questions you want. In short, she tells that these user tests are useless and that nobody is using them.
I beg the differ. Indeed, no one wants to tell the client that nobody understands how to use the product. But is that the user testing fault? No! Somewhere in the process someone had an idea which would be great for the end user. I think that designers, concepters and/or strategists should think more as an end user. These people are able to design and come up with the greatest ideas, but sometimes they lose touch with the purpose of what the client wants. Yes, to make the product nice, functional and pretty is one thing, but making it useful to the end user, is more important. In my opinion, an end user or someone who can represent them should be part in that process, so that his/her opinion can be taken into account in their ideas (=requirements). In this way, an user test would be more appropriate, because the user will test each functionality based on these requirements.
I don’t say that user testing in this form is a must for each product, but you need to understand the meaning of this form of testing. If you understand this, you can decide if user testing is useful for your project or product.
Test your own work
Furthermore, she tells that Fi cannot test their own work. I believe she is talking about user acceptance testing, because Fi has some QA engineers, which are testing the product too.
But user acceptance testing can be done by QA engineers from the producing agency. It is just how you form your QA team. I have worked in and led QA teams which were formed by technical QA engineers, functional QA engineers and acceptance QA engineers. The technical and functional QA engineers are there to test the product and check if everything functions as they should be. Mostly they will do this based on Technical Designs, Functional Designs, Requirements, UX documents, designs or wireframes.
The acceptance QA engineers should test if end users can find their way in the product. They can use the requirements for this or pair up with an end user. A more used technique is exploratory testing. The QA engineer will approach the new product as an end user and records what he/she is doing and if there are defects. Often, this is time-box based (e.g. sessions of 1 hour).
Therefor these QA engineers are not technically educated, but have backgrounds in psychology or marketing. They work on different levels: at the production side, where they help out with the design or concept (as a “virtual” end user) and at the client side (e.g. to discuss issues or support them in making requirements). In this way you can test your own work.
To work like this you need to adjust your recruitment process for QA engineers. You acquire more than 1 profile: a technical one, a functional one and an user acceptance one. Ideally you will hire one of each, but if you cannot arrange this, then consider if a person is capable of doing more than one test process.
QA engineers should be objective in the first place, so in my opinion QA engineers can test the products of their own agency.
Perhaps this sounds as lots of work, but at the end it will be appreciated by the client.
I hope this will set off some discussion in the wonderful world of internet. Feel free to comment or mail me personally.
November 30, 2012
November 29, 2012
November 28, 2012
November 21, 2012