Mercury
Test Assistant for SuperCell
Set different mainstream devices to different image quality (high, medium, and low) based on their configurations (high, medium, and low configuration), and collect performance parameters related to host processes of the game and the mainstream devices during gaming. Then, evaluate the performance of the game with the image quality, and conduct an in-depth analysis on the causes of low image quality. For details, see Figure 2-1.
Process descriptions:
Design test scenarios: Tencent QA engineers select main test scenarios, such as start-up, log-in, matching, battles, and the standby mode, based on features of mobile games.
Select test devices: Tencent QA engineers select three categories (high-end,medium, and low-end) of devices from mainstream devices in the launch regions, ensuring that players using low-end devices can also enjoy smooth gaming experience.
Collect performance data: Tencent QA engineers test different devices in the selected test scenarios and enable automatic collection of performance data, including the frame rate, CPU, GPU, memory, and traffic. For details, see Figure 2-2.
Analysis - CPU: In the rendering process (See Figure 2-3), the overlong CPU calculation increases the time taken to render each frame. As a result, the number of frames rendered per unit time decreases, and users feel that the game is always lagging and choppy. When the CPU usage is excessively high, in-depth data mining needs to be performed to work out the causes (locate the hot function that takes the longest time, and check the reasonability of the function and possibility for optimization).
Analysis - Memory: An in-game memory leak may cause the game to quit unexpectedly, which severely deteriorates user experience (See Figure 2-4). During performance testing, memory leak is tested in key scenarios to prevent the problem. In addition, excessively high memory usage causes the mobile phones to stutter, and in particular, causes the game to fail to operate in low-end devices, which also severely deteriorates user experience. During performance testing, memory data is monitored in key scenarios to find bottlenecks of memory use, and the results are fed back to the developer for optimization and product experience enhancement.
Analysis - GPU: An unstable or over-low FPS may be caused due to GPU rendering. Tencent QA engineers use internal tools to analyze rendering data (meshes, texture, and draw calls) of each frame in the gaming scenario, to check whether resource redundancy exists, whether stickers are too large, or the rendering is excessive, which ca uses players to feel lagging and choppy. For details, see Figure 2-5.
The ultimate goal is to simulate the mainstream configurations of the Internet to cover different experience of Internet users, perform further analysis on the configurations and settings that cause poor experience, to locate performance defects and possible bottlenecks of the game on the mainstream devices in the current market, and feed back the results to the developer to enhance gaming stability and smooth experience.