Agent-based computational economics - Revision history

Xsamm12: /* Possible learning types */

2012-06-20T21:59:09Z

‎Possible learning types

Xsamm12: /* References */

2012-06-20T21:56:01Z

‎References

Xsamm12: /* Other computing methods */

2012-06-20T21:54:35Z

‎Other computing methods

Xsamm12 at 21:54, 20 June 2012

2012-06-20T21:54:16Z

Xsamm12: /* Possible learning types */

2012-06-20T21:53:54Z

‎Possible learning types

Xsamm12: /* Learning */

2012-06-20T21:53:22Z

‎Learning

Xsamm12: /* Agent types and characteristics */

2012-06-20T21:52:17Z

‎Agent types and characteristics

Xsamm12: /* Software and programming */

2012-06-20T21:50:02Z

‎Software and programming

Xsamm12: /* Possible learning types */

2012-06-20T21:49:06Z

‎Possible learning types

Xsamm12: /* Equilibriums and attractors */

2012-06-20T21:46:21Z

‎Equilibriums and attractors

@@ Line 112: / Line 112: @@
 If agent uses anticipatory learning (or [http://en.wikipedia.org/wiki/Temporal-difference_learning temporal-difference learning]), he's trying to predict what might happen in the future, if he takes some action ''A''. Relationship between value functions is therefore recursive. For each possible state it yields the optimum total reward that can be attained by the agent over current and future times. This method requires computation of transtition, return and value functions to compute optimal policy function. These functions are dependent on time and current state. [http://en.wikipedia.org/wiki/Q-learning Q-Learning] enables to compute optimal policy function without knowing these functions. Instead it iteratively acquires the Q-values, that are afterwards stored in observation history. This history is than used to estimate Q-values for next possible action choices<ref name=TesfatsionLearning />.
-[http://en.wikipedia.org/wiki/Cobweb_model Cobweb model] is example of [http://en.wikipedia.org/wiki/Genetic_algorithms_in_economics genetic algorithm for use in economics].
+[http://en.wikipedia.org/wiki/Cobweb_model Cobweb model] is example of [http://en.wikipedia.org/wiki/Genetic_algorithms_in_economics genetic algorithm for use in economics]. For connectionist learning various types of [http://en.wikipedia.org/wiki/Artificial_neural_network Artificial Neural Nets] configurations can be used.
 ==Examples of real applications==

← Older revision		Revision as of 21:56, 20 June 2012
Line 129:		Line 129:
	==References==		==References==
	<references/>		<references/>
−
−	* http://bucky.stanford.edu/numericalmethods/PUBCODE/DEFAULT.HTM
−	* http://www2.econ.iastate.edu/classes/econ308/tesfatsion/ACETutorial.pdf
−	* http://www2.econ.iastate.edu/classes/econ308/tesfatsion/
−	* http://www2.econ.iastate.edu/tesfatsi/aemind.htm
−	* http://www2.econ.iastate.edu/tesfatsi/aintro.htm

@@ Line 126: / Line 126: @@
 ==Software and programming==
 For elaborate overview see [http://en.wikipedia.org/wiki/Comparison_of_agent-based_modeling_software Comparison of agent-based modeling software].
 ==References==

← Older revision		Revision as of 21:54, 20 June 2012
Line 1:		Line 1:
−	'''Agent-based computational economics''' is branch of [http://en.wikipedia.org/wiki/Computational_economics computational economics]. It uses [http://en.wikipedia.org/wiki/Agent-based_model agent-based models] or simulations to model real world market or economic interactions between agents. Agents can represent institutions, firms, individuals or environment. Models, often created in specialized software or framework, are dynamic and allow introduction of heterogenous behavior of agents. ACE is therefore '' "a computational study of economic processes modeled as dynamic systems of interacting agents"<ref name=Tesfatsion2006>Leigh Tesfatsion, Agent-Based Computational Economics: A Constructive Approach to Economic Theory [(pdf,253KB) http://www.econ.iastate.edu/tesfatsi/hbintlt.pdf], in Leigh Tesfatsion and Kenneth L. Judd (eds.), Handbook of Computational Economics, Volume 2: Agent-Based Computational Economics, Handbooks in Economics Series, Elsevier/North-Holland, the Netherlands, 2006.</ref>	+	'''Agent-based computational economics''' or shortly '''ACE''' is branch of [http://en.wikipedia.org/wiki/Computational_economics computational economics]. It uses [http://en.wikipedia.org/wiki/Agent-based_model agent-based models] or simulations to model real world market or economic interactions between agents. Agents can represent institutions, firms, individuals or environment. Models, often created in specialized software or framework, are dynamic and allow introduction of heterogenous behavior of agents. ACE is therefore '' "a computational study of economic processes modeled as dynamic systems of interacting agents"<ref name=Tesfatsion2006>Leigh Tesfatsion, Agent-Based Computational Economics: A Constructive Approach to Economic Theory [(pdf,253KB) http://www.econ.iastate.edu/tesfatsi/hbintlt.pdf], in Leigh Tesfatsion and Kenneth L. Judd (eds.), Handbook of Computational Economics, Volume 2: Agent-Based Computational Economics, Handbooks in Economics Series, Elsevier/North-Holland, the Netherlands, 2006.</ref>
	''		''

@@ Line 83: / Line 83: @@
 ===Possible learning types===
-There are various types of learning algorithms <ref name=TesfatsionLearning />). Here is brief summary by Leigh Tesfatsion:
+There are various types other of learning algorithms suitable for use in ACE <ref name=TesfatsionLearning />). Here is brief summary by Leigh Tesfatsion:
 # Reactive Reinforcement Learning (RL)
 ## Example 1: Deterministic reactive RL (e.g. Derivative-Follower)

@@ Line 75: / Line 75: @@
 ===Learning===
-In order to capture dynamic nature of real markets agents should be able to learn which means change their behavior according to the situations they encounter. (zdroj) Agents in ACE can use various types of learning algorithms. Selection of an algorithm can fundamentally influence the results of the simulation<ref name=Tesfatsion />. [[Roth-Elev]] [http://en.wikipedia.org/wiki/Reinforcement_learning reinforcement learning] algorithm is one of the possible choices:
+In order to capture dynamic nature of real markets agents should be able to learn which means change their behavior according to the situations they encounter. (zdroj) Agents in ACE can use various types of learning algorithms. Selection of an algorithm can fundamentally influence the results of the simulation<ref name=Tesfatsion />. [[Roth-Elev]] [http://en.wikipedia.org/wiki/Reinforcement_learning reinforcement learning] algorithm is one of the possible choices. It works in following steps:
 # Initialize action propensities to an initial propensity value.
 # Generate choice probabilities for all actions using current propensities.

@@ Line 72: / Line 72: @@
 ==Agent types and characteristics==
-Simple programmed agents are represented by simple algorithm, be it short lenght of a code or simplicity of a pseudo-random number generator agent uses <ref name=Chen />. However even simple agents can exhibit form of swarm intelligence simillar to the emergent behavior of a group ants or termites. Groups of simple agents are than capable to solve complex tasks. Even without learning capability, the agents can optimize or generate orderly movement patterns. [http://en.wikipedia.org/wiki/Stigmergy Stigmergy] can be one way to achieve this.  Agents can be differentiated by position in the [http://economistsview.typepad.com/economistsview/2009/02/cognitive-hierarchy-theory.html cognitive hierarchy], where more complex agents are able to think more steps ahead than simple agents. Smarter agents can also emulate behavior of simple agents if favourable but it's not possible vica versa. Non-agent economic models often introduce simplifying assumptions e.g. that all agents are rational and homogenous <ref name=North>Charles M. Macal and Michael J. North, "Tutorial on Agent-Based Modelling and Simulation", http://www.econ.iastate.edu/tesfatsi/ABMTutorial.MacalNorth.JOS2010.pdf , Journal of Simulation, Vol. 4, 2010, 151–162</ref>. Humans interacting in various systems or institutuins are heterogenous and it's desirable to emulate this feature to produce more realistic behavior<ref name=Chen />.
+Simple programmed agents are represented by simple algorithm, be it short lenght of a code or simplicity of a pseudo-random number generator agent uses <ref name=Chen />. However even simple agents can exhibit form of swarm intelligence simillar to the emergent behavior of a group ants or termites. Groups of simple agents are than capable to solve complex tasks. In some cases even without learning capability, the agents can optimize or are able to generate orderly movement patterns. [http://en.wikipedia.org/wiki/Stigmergy Stigmergy] can be one way to achieve this.  Agents can be differentiated by position in the [http://economistsview.typepad.com/economistsview/2009/02/cognitive-hierarchy-theory.html cognitive hierarchy], where more complex agents are able to think more steps ahead than simple agents. Smarter agents can also emulate behavior of simple agents if favourable but it's not possible vica versa. Non-agent economic models often introduce simplifying assumptions e.g. that all agents are rational and homogenous <ref name=North>Charles M. Macal and Michael J. North, "Tutorial on Agent-Based Modelling and Simulation", http://www.econ.iastate.edu/tesfatsi/ABMTutorial.MacalNorth.JOS2010.pdf , Journal of Simulation, Vol. 4, 2010, 151–162</ref>. Humans interacting in various systems or institutuins are heterogenous and it's desirable to emulate this feature to produce more realistic behavior<ref name=Chen />.
 ===Learning===

@@ Line 68: / Line 68: @@
 Model behavior can result to various types of equilibrium and attractors. '' "System is in equilibrium if all influences acting on the system offset each other
 so that the system is in an unchanging condition" ''
-<ref>http://dl.acm.org/citation.cfm?id=1531270</ref>. Agent-based models can help to determine which parameters influence stability or effectiveness of the market while visualization capabilities can help to identify possible [http://www.scholarpedia.org/article/Basin_of_Attraction basins of attraction]. Agent can for be for instance attracted by different basins of attraction while using different learning algorithms. Image on the right shows how agent scale the profit curve using deterministic reactive reinforcement [[#Learning| learning]]. Because of using simple Derivative-follower adaptation<ref name=TesfatsionLearning /> agent stops when profit level start's to fall, which is in this case too soon. Parameters can be changed on different levels e.g. agent level, market level or world level. Agent may have parameters like risk aversion, market may have parameters like non-employment payment percentage etc.<ref name=Tesfatsion></ref>
+<ref>http://dl.acm.org/citation.cfm?id=1531270</ref>. Agent-based models can help to determine which parameters influence stability or effectiveness of the market while visualization capabilities can help to identify possible [http://www.scholarpedia.org/article/Basin_of_Attraction basins of attraction]. These can than be pinpointed through generated reports, plots or through other available ex-post analytical tools. Agent can for be for instance attracted by different basins of attraction while using different learning algorithms. Image on the right shows how agent scale the profit curve using deterministic reactive reinforcement [[#Learning| learning]]. Because of using simple Derivative-follower adaptation<ref name=TesfatsionLearning /> agent stops when profit level start's to fall, which is in this case too soon. Parameters can be changed on different levels e.g. agent level, market level or world level. Agent may have parameters like risk aversion, market may have parameters like non-employment payment percentage etc.<ref name=Tesfatsion></ref>
 ==Agent types and characteristics==