How do schemas develop

Schemata and sequential thought processes in PDP models. McClelland, D. Explorations in the microstructure of cognition Volume 2: Psychological and biological models, pp. Seel, N. Mental models and problem solving: Technological solutions for measurement and assessment of the development of expertise. Blumschein, W. Hung, D. Strobel Eds. Rotterdam: Sense Publ. The use of schemas as a basic concept was first used by a British psychologist named Frederic Bartlett as part of his learning theory.

Bartlett's theory suggested that our understanding of the world is formed by a network of abstract mental structures. Theorist Jean Piaget introduced the term schema, and its use was popularized through his work.

According to his theory of cognitive development , children go through a series of stages of intellectual growth. In Piaget's theory , a schema is both the category of knowledge as well as the process of acquiring that knowledge. He believed that people are constantly adapting to the environment as they take in new information and learn new things. As experiences happen and new information is presented, new schemas are developed and old schemas are changed or modified.

For example, a young child may first develop a schema for a horse. She knows that a horse is large, has hair, four legs, and a tail. When the little girl encounters a cow for the first time, she might initially call it a horse. After all, it fits in with her schema for the characteristics of a horse; it is a large animal that has hair, four legs, and a tail.

Once she is told that this is a different animal called a cow, she will modify her existing schema for a horse and create a new schema for a cow.

Now, let's imagine that this girl encounters a miniature horse for the first time and mistakenly identifies it as a dog. Her parents explain to her that the animal is actually a very small type of horse, so the little girl must at this time modify her existing schema for horses.

She now realizes that while some horses are very large animals, others can be very small. Through her new experiences, her existing schemas are modified and new information is learned. While Piaget focused on childhood development, schemas are something that all people possess and continue to form and change throughout life.

Object schemas are just one type of schema that focuses on what an inanimate object is and how it works. For example, most people in industrialized nations have a schema for what a car is. Your overall schema for a car might include subcategories for different types of automobiles such as a compact car, sedan, or sports car.

Other types of schemas that people often possess include:. The processes through which schemas are adjusted or changed are known as assimilation and accommodation. In assimilation , new information is incorporated into pre-existing schemas. Schemas tend to be easier to change during childhood but can become increasingly rigid and difficult to modify as people grow older. In their work, OACs are learned through a supervised learning method and are tested on a robotic arm. The goal of the experiment they presented is to move an object from one point to another by removing obstacles along the path.

Their results show that the OACs model is capable of planning and making predictions. However, goals for the planning are set up by the user rather than the agent itself. This limits the agents capability for performing open-ended learning and encouraging continuous play behavior. Moreover, the capability to generalize experiences is recognized as future work, limiting its performance within novel environments.

Learning consists of schemas that contain preconditions, an action and the postconditions, which are results of applying the action on the preconditions. The model is employed by a virtual agent in a 3D virtual environment, where it can passively observe the environment by moving its head and by fixating to interesting objects. The latter are found by the use of an attention process based on an interest value of the perceived objects. Interest values depend upon three aspects; pre-programmed preferences, number of object features properties and virtual emotional interest in the object.

The agent is initially provided with two reflexive saccade schemas, through which it develops its knowledge by interacting in the environment to create more schemas. The attention system helps the agent to demonstrate the playful behavior.

However, the model is not capable of planning in order to achieve a goal within the environment, or to exploit a sequence of actions in order to achieve a state in the environment which is not possible with a single action. Most recently, Kansky et al.

The objects are represented by lists of fixed binary properties, where an object may or may not have a given property in the environment. The network can perform planning toward maximizing the reward from the initial state as it matches the goal state in the environment.

To evaluate the network, an experiment is performed using the environment of the classic arcade video game Breakout. In the game, a ball is used to gradually break a brick wall positioned at the top of the screen by being repeatedly bounced between the bricks and the player's paddle that moves horizontally on the bottom of the screen.

Points are awarded every time a brick is hit by the ball, which is enough to break it from the wall, without missing the ball. The performance of the schema network is compared with two other deep learning network models; Asynchronous Advantage Actor-Critic A3C and Progressive Networks PNs , in different experiments containing different variations in the environment. The results show that the proposed network outperforms the others in all the variations of the environment, capable of generalizing and adapting what it has learnt to variations of the environment.

However, the network still needs a large amount of training to achieve a better result. The above learning models are comparable to the schema-based mechanism proposed in this work. Here, we present an intrinsically motivated open-ended learning and play generator system, termed Dev-PSchema. By employing it, an agent plays and learns that a ball can be grasped and moved to a different location and disappears when dropped in a hole. The system can learn from a small number of experiences and can combine them in order to construct higher level reusable chains of actions to represent more complex hierarchical behaviors.

An excitation mechanism triggers learning by exploratory play during which the system generalizes schemas and re-uses them in novel situations. Moreover, with a change in the excitation parameters, different individual infants are simulated, a feature that is absent to all of the above discussed learning models.

Finally, the system is sufficiently abstract and can be used with different platforms without making any major design changes. In Dev-PSchema, each schema consists of the pre and post states of the environment i.

The work presented in this paper draws inspiration from Piaget's schema mechanism. An initial implementation of this mechanism is given in Drescher , with a model based on the sensorimotor stage, i. Learning at this stage is believed to be associated with motor actions that are performed by the developing infant. Based on this idea, the schema system simulates an agent which learns from its sensory experiences that result from motor actions, and uses the knowledge that was previously acquired to interact with the environment.

The mechanism has no concept of persistence of objects while associating the sensory cues, i. In Section 2 we present Dev-PSchema and the experiments along with the results. In Section 3 we discuss the system's capability to express different behaviors due to variations in the excitation parameters and to learn high-level actions by developing schemas chains.

Finally, in Section 4 we provide a conclusion about our findings in the light of developmental psychology. Dev-PSchema builds on PSchema, a previously developed system by Sheldon , and simulates an agent within an environment capable of interacting with it.

By considering simulated sensory information as well as actions that the agent can perform, the system is capable of learning action-effect correlations. These are represented as schemas and constitute the knowledge the agent gains by interacting with objects within the environment. At the beginning, the system starts with a basic set of action schemas, referred to as bootstrap schemas details are found in Kumar et al. Subsequently, the system is free to start applying the schemas in the environment and, by interacting with objects, to learn new ones while expressing playful behaviors.

As such, the system is considered a play generator that allows infant behaviors and learns to emerge through playing. As the agent interacts with the environment, new schemas are added to record new experiences or unexpected outcomes from actions, incorporating the preconditions from which the effect was experienced. These new schemas contain a set of sensory information, the behavior and its predictions in the environment. We refer to the sensory information as preconditions, the behavior as action and the sensory predictions or results as postconditions.

Thus, a schema is a tuple that consists of an action and the sensory information from both before and after the execution of the action, as preconditions and postconditions respectively. Any unpredicted effect of actions, as described by the schema used by the agent at any time, leads to the generation of new experiences that are also captured as new schemas. For instance, this happens when the postconditions of a schema do not match the resulting phenomena of the schema's action.

Note that Dev-PSchema operates in discrete time; the system records observations before and after the execution of an action. Counting actions that are performed from the beginning of an experiment indicates the time-steps. During a single time-step, the system records all available observations to form the preconditions, executes an action and finally records observations again to form the postconditions.

A chain of schemas is also executed within a single time-step. Table 1 shows an example of a schema that was learnt after grasping an object using an initial bootstrap schema.

Here the sensory information and the actions are defined as high-level abstractions, rather than the sets of raw sensor data and motor commands that they reflect. When in use, the system is connected to a body 1 via a low-level system that is responsible for the generation and availability of perceptions and actions for the schemas. In the case of real robotic hardware, the low-level system translates the schema actions into appropriate motor activities allowing the agent to interact with the environment.

Although schemas could be used to represent low level actions and sensory information the focus here is on high level playful behavior. In order to generate play behaviors within an environment, attention and novelty are important Mather, Dev-PSchema employs an excitation mechanism that provides action selection by identifying those object-action pairs that are most interesting to the agent considering their postconditions.

Selection of interesting object-action pairs depends upon the agent's preferences. Whereas such preferences are affected by novelty and habituation i. The system provides exploratory play behaviors to interact with the objects and learn outcomes related to different actions performed on them. Note that the objects in the system are defined with the visual perceptions containing underlying properties. On one hand, the system is capable of exploring an object by performing actions associated with it.

On the other, the system has the ability to switch between objects as necessary, ensuring the evaluation of the transferability of any learned knowledge while encouraging further explorations. Furthermore, the system is able to create sequences of schemas in order to achieve a distance state i. The agent will create new schemas and chains of schemas from existing schemas wherever possible following the execution of a schema or chain. The process of creating new schemas following interaction resembles the adoption process where a subject learns new knowledge building upon an existing knowledge base as described by Piaget and Cook Below we describe the key components that allow the generation of schemas and schema chains and therefore the development of the learning.

In particular the excitation calculator Section 2. Considering all objects in the environment, as they are perceived via sensory information, the agent calculates the excitation of each available schema in order to find the most interesting one to be executed with respect to the current perceived environment referred as world state.

Calculating the excitation is based on the similarity, novelty and habituation assigned to each schema, the total excitation of a schema is a weighted combination of these three factors.

Varying the weights allows the generation of different play behaviors Oudeyer et al. In particular, similarity is designed to favor schemas related to previous interactions with a given object, whereas novelty increases the excitation value for new objects or objects that have not been interacted. Subsequently, habituation decreases the interest the agent has for an object that is frequently used for interactions over time.

Obviously, novelty and habituation are in contradiction by which the agent switches its attention from objects that have been explored to those that propound novel interactions. Note that although the terminology used in this work is based on that of developmental psychology, the meaning is not an exact match.

Therefore, a precise definition of all three of such factors of excitation are given below. This factor is used to describe the degree of resemblance between the object-specific perceptions that are captured at the end of an action and those that constitute the postconditions in each of the previously learned schemas. It is calculated by matching individual properties of an object, such as color or shape.

If a property appears in both states but the values are different, then Sim will return a partial match, i. The result, in short, is the ratio between the sum of all maximum similarities calculated by Sim and the total number of properties in the perceived object. The result is a number between 0 and 1, with 1 indicating an exact match. Although each property is compared with all properties found in all schemas, only the one with the maximum similarity measure is considered.

This is calculated by considering how frequently perceptions that describe an object are confirmed as postcondition in schemas, in connection to the running time-step:. Initially the novelty of the newly perceived object will be the maximum. As the object is played with more frequently or appears more in schemas its novelty reduces. If the object is not played with for a longer period of time, its novelty again increases. Figure 1. This factor depends on how recently schemas containing the object perception are used in the environment.

The agent is expected to be more habituated, hence less interested, with a situation that reoccurs after interacting with the environment. This is inspired by developmental psychology, where infants become habituated with objects or events after a period of exploration or observation Sigman, ; Hunter et al. Habituation at a given time-step is given by.

Thus the overall habituation is computed by. Figure 2. Similar to novelty, the coefficient at the exponential is designed to smooth the curve for the range 0—1. Habituation is expected to increase as frequent interactions with the environment lead to the same object perceptions being captured, which in turn allows the agent to select actions that promote interactions with different areas of the environment.

The total excitation is calculated by combining similarity, novelty and habituation, such that. This allows the agent to select an appropriate object to interact with, by utilizing previous experiences associated to all objects in the environment. In particular, novelty and habituation are directly combined as they are both related to experiences associated with the currently perceived object, whereas the similarity considers all experienced perceptions of the objects which the system has previously interacted with.

By varying the weights, we can simulate different artificial infants with different preferences e. This can also be seen as a preference toward exploration or exploitation. Alongside the object-related excitation, the agent calculates the excitation of each schema in the system, in order to select an appropriate schema to be employed.

Thus, this excitation is related to the possible actions that could be performed for each object, rather than the object perception alone. If the perception s in the environment following an action matches the post conditions of the schema, the execution is considered to be successful. A success rate S r is maintained to record the proportion of time that the expected outcome of a given schema has been achieved. This can also be considered as a reliability measure for each schema, such that.

A coefficient to the exponential power is used as a smoothing factor to obtain an exponential response over the values of the ratio between schema executions and current time. Ultimately, the final excitation for each schema is calculated by considering each object that is present in the environment, so that.

In a similar vein, schemas that are never used become more excited than their recently executed counterparts, enabling the agent to explore the environment by performing different actions.

Algorithm 1 describes the process of calculating the excitation for a given environment state, referred to as world state WS in the system. It computes the excitation of schemas and schema chains to be introduced next and returns the schema or chain with the highest excitation, following the winner takes all principle. In the case of equal excitation, schema chains will be preferred to encourage the system to explore more complex behaviors.

During the calculation of schema excitations, the system generates schema chains as described below in Section 2. Once finished, Algorithm 1 results to the schema or chain with the highest excitation. As an agent gains more experiences and skills, certain skills can be linked together to form higher level skills in a hierarchical structure. Through playful exploration, more complex chains can be learned that combine basic and form more sophisticated high level actions.

Chains are seen as sequences of schemas, which the agent discovers by finding the links between the preconditions and postconditions of the schemas in memory. Chaining helps in achieving distant states of the environment that are not possible when employing a single schema.

For example picking up an object from a reachable position needs two different actions to be achieved; i reach for the object and ii grasp it. Figure 3 shows an example of a two schema chain obtained by linking the preconditions and postcondition of two different schemas.

Figure 3. An example of chaining. Algorithm 2 is responsible for the chain generation. As previously mentioned, chains are created during the process of calculating the excitation for schemas. Longer chains are discouraged during the chaining process in order to reduce computational costs and avoid overly complicated chains that are more likely to be unsuccessful.

Here, a limit of 5 schemas is set. In Algorithm 2, the schemas S s contains preconditions which are a subset of the current environment, WS. The algorithm adds all the possible chains, for a given state of the environment, into the memory and returns the most reliable chain among them. Reliability of a chain is calculated by taking the average of success probabilities of all the schemas present in the memory.

Schemas in a chain are executed in a sequential order. A chain is considered successful if the resulting WS due to the preceding schema's action matches its postconditions. A chain execution is performed either as chain reflexes or motor programs as described below. Initially chains are executed in the chain reflex mode. The world state sensory information from the environment is considered at the end of every executed schema in the chain.

If it does not match the expected postconditions of the executed schema then the schema chain is considered unsuccessful. An unsuccessful chain is then opted out from the next step's schema selection. If a chain is successfully executed multiple times, then it is considered reliable and therefore becomes automatic, in a sense that it behaves as a singular continuous higher-level action called a motor program.

As such, the chain is used to achieve a certain condition that results from a hierarchy of actions. Motor programs are executed sequentially without the need of intermediate verification of the world state. That is, only the last action's resulting postconditions are used for the evaluation of the motor program.

Algorithm 3 describes the execution process of an exciting schema or a chain. Note that executing a chain is considered as taking a single time step. For further details on this mechanism of Dev-PSchema, please see Kumar et al. During the execution of a motor program, although the external state of the environment may not be directly monitored by the high-level agent, the internal proprioceptive system is active.

When interfaced with a low level system that is monitoring all the sensors, the chain can still be interrupted if something unexpected was perceived. The concept of schema chains is inspired from developmental psychology, where the ability for planning, hence action sequences, is investigated Willatts and Rosie, ; McCarty et al.

McCarty et al. A spoon full of food was placed in various orientations in front of the infant. It was observed that 9 and 14 month old infants reached and grasped the spoon with their preferred hands. Due to difficult orientations of the spoon, 9 month old infants were found to grasp the spoon from the opposite side of the spoon, i.

The 14 month old infants always made corrections to make sure that the food reaches the mouth, whereas the 19 month old infants were found to switch to their non-preferred hand when the orientation of the spoon was difficult. The authors identified a series of planned strategies employed by the infants each with the goal of eating the food that can be considered as chains of action schemas. Gender Differences.

Cultural Differences. Psychological Reactance Key Takeaways. Liking and Loving Chapter Learning Objectives. Initial Attraction Learning Objectives. Physical Attractiveness. Gender Differences in Perceived Attractiveness. Why Does Similarity Matter? Status Similarity. Closeness and Intimacy. Communal and Exchange Relationships. Interdependence and Commitment. What Is Love? Individual Differences in Loving: Attachment Styles. Making Relationships Last.

When Relationships End Key Takeaways. Helping and Altruism Chapter Learning Objectives. Brad Pitt Helping in New Orleans. Reciprocity and Social Exchange. Positive Moods Increase Helping. Taking Responsibility. Implementing Action Key Takeaways. Other Determinants of Helping Learning Objectives.

Gender Differences in Helping. Who Do We Help? Attributions and Helping. Reactions to Receiving Help. Cultural Issues in Helping. Increasing Helping Key Takeaways. Thinking Like a Social Psychologist about Altruism.

Aggression Chapter Learning Objectives. Star Striker Banned for Biting Again. Defining Aggression Learning Objectives. Is Aggression Evolutionarily Adaptive? The Role of Biology in Aggression. Hormones Influence Aggression: Testosterone and Serotonin. Drinking Alcohol Increases Aggression. Individual Differences in Aggression.

Gender Differences in Aggression. Thinking Like a Social Psychologist about Aggression. Gender Diversity in the Workplace. Understanding Social Groups Learning Objectives. Communication, Interdependence, and Group Structure. Social Identity. Group Performance Learning Objectives.