The information economy will be by far the largest multi-agent system ever envisioned, with numbers of agents running into the billions. Building economic behavior into the agents themselves offers a twofold promise: their myriad interactions and conflicts will be governed by the same economic principles that have lent coherency to the activity of billions of humans; and they will be able to operate in the same economic space as humans, to the benefit of both species. But there are potential pitfalls too. Economic software agents differ from economic human agents in significant ways, and their collective behavior may not closely resemble that of humans. Software agents must be designed with the understanding that they will be operating in (and helping to create) complex nonlinear dynamical environments.
Even before the advent of the full-fledged information economy, automated pricing of physical goods such as books will be an interesting and important topic in its own right. Our hope is that, by exploring the nonlinear, collective dynamics of automated posted pricing and by focusing particularly on the interplay among learning, optimization, and dynamics, we may obtain insights that are both intrinsically useful and more generally applicable to other aspects of agent-based information economies. Naturally, it will be important to study the relationship between individual agent strategies and collective market dynamics for one-on-one negotiation and auctions of all varieties. Beyond negotiation lie other important collective issues such as agent reputations. Some of the qualitative lessons that we have drawn from our studies of dynamic posted pricing seem likely to apply in these realms as well.
One general point is that agents will have to learn, adapt, and anticipate. In order to do so they will use a variety of machine learning and optimization techniques. Much of the work on machine learning and optimization has implicitly assumed a fixed environment or opponent. But agent economies are guaranteed to be dynamic by virtue of the fact that the agents are all learning. This violation of standard assumptions has important consequences.
For example, ordinary single-agent Q-learning is guaranteed to converge to optimality. When we introduced Q-learning into the shopbot model we found that two learners could fail to reach convergence (although they still reached a mutually beneficial state). Understanding the dynamic interactions among a society of learners is of fundamental theoretical and practical interest, and only a few beginning efforts have been made in this area [31, 15, 28, 26]. A second example was our use of the popular amoeba optimization technique as a means by which an information bundler might optimize its profit. Here, the amoeba's implicit assumption that it was optimizing a static function caused it to fail miserably. The reason it was not optimizing a static function was that other agents in the system (the buyers in this case) were learning. In this case, it was possible to rectify amoeba by having it periodically re-evaluate its solutions. This resulted in short-term price volatility but long-term profit stability. It remains to be seen whether such a technique will be sufficiently adaptive to work against competition.
Another general lesson is that plausible agent strategies can lead to both beneficial and harmful collective behaviors. Economic incentives can drive the consumers and brokers in the information filtering economy to a niche monopoly in which the consumers' utilities and brokers' profits are both quite high. However, all three models in which we allowed for multiple sellers (all but the information bundling model) were vulnerable to cyclical price-war dynamics. In the shopbot model, the myoptimal sellers were somewhat hurt by the moderately low average prices. In the information filtering model, the consumers were hurt as well because the cycling in price/product space caused brokers to ignore all but the most profitable market segments. The cycling can be eliminated or ameliorated by machine learning techniques that allow agents to account for the future consequences of their present actions. We are currently investigating how to make these techniques practical for more than two sellers. Cycling behavior can also be reduced somewhat if the sellers are not sufficiently capable of computing (myopically) optimal decisions. This is likely to occur when information about buyers is hard to obtain, or when optimality is difficult to compute. In the first case, techniques such as amoeba can enable a seller to learn aggregate buyer information (at least in a monopoly). The second case is most likely to arise when the seller is setting product parameters as well as prices (as in the information filtering model) or several price parameters simultaneously (as in the information bundling model).
It would be imprudent to use the world's economy as an experimental testbed for software agents. Our approach will continue to be to use the familiar tools of modeling, analysis, and simulation to study and redesign agent strategies, protocols, and market mechanisms in the laboratory before releasing agents and agent infrastructures into the world's economy. An especially exciting aspect of this work is that it requires us to go beyond traditional tools and techniques, and leads us into new realms such as multi-agent learning in which new fundamental scientific developments and breakthroughs are required. We eagerly anticipate creative contributions from researchers in many fields, particularly economics, computer science, and applied mathematics.