The Stack reports on a talk by Professor Giovanni Vigna concerning the increasing sophistication of malware in its attempts to evade detection by effectively laying low on a target system. Detection is the enemy of malware, because it is quickly followed by updates to anti-virus software to enable detection of its signature and subsequent removal from the ecosystem. Of particular importance to the creators of malware is that it does not become active in a virtual machine of the kind used by anti-virus software to encourage safe activation. Malware therefore needs to do two things that work against one another: to evade detection by staying quiet, and to deliver its payload by becoming active.

This is a very common problem in decision strategy when participants' interests are not aligned, and when there is relevant but hidden information. Under these conditions, the interaction becomes what in game theory parlance is known as a 'signalling game'. In the situation described above, there are actually two simultaneous signalling games going on: the malware is trying to signal that it is benign, and the anti-virus's virtual snare is trying to signal to the malware that it's a real machine ready to be infected.

Signalling games occur everywhere. Interviewees want to appear more competent than they are. Men and women want to appear more attractive than they are to potential suitors. Manufacturers want their goods to look high-quality even when they aren't. Both China and the US want to signal resolve over Taiwan and avoid revealing the true price they'd be willing to pay for their objectives. The economist Robin Hanson holds the view, with considerable conceptual support, that signalling is the main driver of most human behaviour. The game of poker - among many others - is fundamentally a signalling game; players with strong hands want to look mediocre, and players with weak hands want to look strong.

The theory of signalling games is a surprisingly recent invention, generally considered to have begun with George Akerlof's 1970 article 'The Market for Lemons'. Signalling games can resolve one of three ways. 'Pooling' and 'separating' occur when everyone transmits either the same signature or unique signatures respectively. The more-interesting mixed outcome is when only a proportion of players send the 'false' signal. As a general rule, 'good' signal senders (non-malicious software) and receivers (anti-virus software) would like separating solutions in which they can identify one another. 'Bad' senders (malware) would like to be in a pooling or mixed solution in which they reap the rewards that should be due only to the 'good' players.

It turns out that in these situations a key determinant is signalling costs, and specifically whether or not the receiver can impose a higher cost on 'bad' types than on 'good' types. This kind of solution occurs everywhere: it's easier for a clever person to get a good degree than a stupid one; it's easier for an attractive person to look nice than an ugly one; it's easier for a manufacturer of high-end cars to demonstrate their quality than a firm smashing out bangers; it's easier for a competent person to pass an interview than an incompetent one. Good malware design, and good anti-virus detection, will take account of this by trying to find ways to make it costlier for virtual machines to pretend to be real, or for malware to look benign.