Summary of Research in Learning Design

27 11 2013

Summary of Research in Learning Design

This is by no means a comprehensive summary but more of an in-progress, first impressions summary of research developments in the field of learning design, which I will add to as I read and discover more.

Aims (aspirations) of Learning Designers & Learning Design Research (LDR):

  • Core concepts of LD are: guidance, representation and sharing (LDCM)
  • Making design processes more explicit and shareable (Conole, 2013)
  • Creating more effective learning environments and interactions for learners with the aim of assisting effective and meaningful learning.
  • New approaches are needed to help educators make effective use of technologies + educators often not experienced in design.
  • To help educators move from practical consciousness (we do it this way) to discursive consciousness (I am doing this because) based on Giddens, from OULDI-JISC report. (Note this is what HEART project is aiming to do).
  • “development of tools, design methods and approaches to help teachers design pedagogically effective learning activities and whole curriculum which make effective use of technologies.” (Conole, 2013, p.6.)
  • One aim is to generalise across cases, streamlining the process of future design by offering general principles of application, or even universal patterns (Beetham & Sharpe 2007).
  • 2 goals (Bennett et al 2009): to provide a means for teachers to document and work with their designs for planning and implementation; and to provide a way of sharing, adapting and reusing designs. “The aim is not to prescribe a particular design to be copied but to extend a teacher’s pedagogical repertoire through the process of modifying someone else’s design.” P.155

Overview of Learning Design Research
It appears that within the field of Learning Design there was an initial “sprouting” of a variety of different approaches, initiatives and tools which aimed to find ways to represent and share learning designs (e.g. IMS LD, LAMS, LDSV, Phoebe, London Pedagogical Planner/Learning Designer, Pedagogical Patterns Collector, Compendium LD.  See Larnaca Dec website for a summary and timeline of these).  Agostinho et al (2008) categorised these “LD languages” into 4 categories:

  • Pedagogical patterns (Phoebe)
  • Generic lds i.e. patterns & generic ld visualisation sequences (LDVS)
  • Contextualised ld instantiations (LDVS, LD Lite and E2ML)
  • Executable, runnable versions (IMS LD, LAMS)

Some of these sprouts withered and died and others have survived and over time there was a discovery by researchers that practitioners don’t necessarily use the learning designs (either at all, or in the way intended).  This limited uptake and sharing of learning designs ‘not invented here’ (Philip& Cameron, 2008) then led to further research into why this might be the case.

One strand of research looked at whether a LD can be consistently and clearly represented.

A literature review of international research from 2004-2008 by Agostinho et al 2009 found that effective lds share three elements:

  • clear and explicit description of pedagogy of ld
  • some form of quality rating of the ld (eg evaluative findings)
  • explicit guidance/advice about how the ld could be used

Agosthinho et al (Ch 6 in Beetham & Sharpe, 2013) also identified a need for embedding design support in LMSs but noted technical difficulties in implementing this.  These authors are now planning to carry out further research into lds as a stimulus for design thinking (using case-based reasoning).

Masterman (2013) also addresses this issue and concludes that there are 3 ongoing issues for representing learning designs:

i. providing computational support for cognitive actions in an “ill-defined” domain
Issues include: how to design guidance for educators that is useful without being overly constraining, not too simplistic, nor too vague/general; the elusive nature of the early stages of the design process when many things are being shuffled and managed in rapid iteration (often with paper/pen or mind-mapping tools) – this is difficult to capture.
ii. Positioning the tools vis-à-vis theory
Given the multiplicity of theoretical influences on teachers Masterman suggests a supportive digital tool that is rooted in “a principled theory of educational processes” is needed rather than tool that is built on a prescriptive framework. (What does she mean by this?)
iii. Politics of deployment
Masterman identifies several issues in the deployment or implementation of lds including: tensions identified in design itself (method vs creativity); theory-informed vs pragmatic approaches; the ill-defined nature of LD vs systematisation into an ontology to underpin computational support; conformity vs creativity in educational practice.

Conole (2013) refers to Gibbons & Brewer’s (2005) summary of the dimensions of design languages which provides a good summary of the challenges inherent in representing design:

  • complexity – design is a partial representation of a complex multifaceted ideas
  • precision – tension between natural, fuzzy, real practice and highly defined specification
  • formality & standardisation – do the terms used mean the same thing to all users
  • tension between personally created designs and those shared with others
  • designs are dynamic, not static artefacts & co-constructed in context
  • issues around computability.

Falconer & Littlejohn (2008) identified an overlapping set of challenges for LD representation:

  • Ownership of lds
  • Representations need to fulfil multiple purposes
  • Generic vs detailed i.e. used for orchestration or inspiration
  • Designs are both product and process
  • Degree of granularity

The recent development of the LDCM has provided a conceptual framework in which to place these issues though it has not necessarily solved them.  Challenges such as the degree of granularity can be relatively easily managed by careful and consistent use of language.  Other challenges such as the inherent complexity of design, the fuzziness of real practice,  need for representations to fulfil multiple purposes and whether they are used for orchestration or implementation are in reality elements of tension in Learning Design research and practice, which researchers and practitioners need to be aware of.  As the field continues to develop and mature researchers and practitioners are likely to find ways of managing these tensions within research and practice. Ongoing research and development of how to represent learning designs is continuing in multiple contexts. The field as a whole continues to work towards developing a widely acceptable descriptive framework for l & t activities (or “educational notation”.(Larnaca declaration, p.5)  Though, as the Larnaca Declaration notes “If a notation system (or systems) for describing teaching and learning activities is developed and widely adopted, its success will be due to a complex mixture of its accuracy, expressiveness and historical contingencies.” (p.31)

A second and connected strand of research has focused on exploring educators’ actual design practice in more depth to work out whether any insights could be gained from authentic design practice that could then feed back into learning design models and approaches.  This strand overlaps with the first and often involves the same researchers but the focus is more on the guidance and sharing aspects of LD rather than its specific representation.  This strand explores questions and issues around how to help educators to describe, share, adapt and reuse lds.  It is about designing for learning (Beetham & Sharpe, 2013) or Learning Design Practice.

Agostinho, (Ch 7 in Beetham & Sharpe, 2013), found that:

  • lds (using LDVS representation method) could be readily understood and used
  • lds are used for design ideas ie as a source of ideas rather than a model to replicate
  • lds are used for benchmarking i.e. teachers use to assess their previous and current work and build confidence. In this sense lds are models of good practice not descriptive pedagogy.
  • designing with a ld led to an improvement in teachers pedagogical content knowledge (PCK, Shulman 1980) and TCPK (Mishra & Koehler, 2006).  TCPK follows (or perhaps is secondary) to PCK.
  • teachers prefer contextualised examples (where the design is described in its original context) rather than a more generic guide. Contextual detail appears to add to useability.  This finding raises questions concerning the usefulness of more generic designs.

Another study by Agostinho in 2011 found that the LDVS (from the 2003 AUTC project) was used to document teaching ideas, as a design tool to discuss teaching ideas with colleagues, or as an analysis tool to reflect on learning design.

Oliver et al (Ch  6 in Beetham & Sharpe 2013) found that the LDVS framework with tasks, resources and supports provides an efficient means to describe learning experience and to represent teachers plans but noted issues with:

  • Teachers come up with different ways of representing designs, even when using the same framework.
  • Difficulty identifying the critical elements – teachers describe things in different ways.
  • Varying size and scope of LDs, which makes comparison difficult.

Some researchers’ attention has turned to finding effective means to portray and communicate the pedagogical frameworks and relevant contextual information to support the reuse of learning designs (Bennett et al., 2007; Philip & Cameron, 2008, in Donald et al 2009).

Masterman (Ch 4 in Beetham & Sharpe, 2013) concludes that the design process is complex, related to individual dispositions of practitioner and a range of intellectual and socio-cultural influences and the nature of the process itself. She suggests that to provide design support and tools we need to understand teacher’s design practice and setting, particularly:

  • Conceptualisation of and approach to activity of design:
    • Design is both systematic and creative, structure emerges from fluidity and negotiation
    • Tension/distinction between planning (laying out of constraints) and design (what can be achieved within constraints)
    • Design is a messy, juggling process – teachers start at different places, take different route and employ different metaphors, modules and visualisations
  • Factors that bear on teacher’s approach to design
    • These include style/approach of teaching they experienced, institutional polices and structures, student related factors, theory, research-informed thinking, nature of the discipline.
    • Theory can both inform the activity of creating a ld and fulfil an explanatory function in the activity of reviewing (reflection).
    • Practitioners tend to draw on number of theories, some take a-theoretical stance. Pragmatism often wins out.
    • There is an increasing awareness of the value of evidence based practice to design.
    • There are differences between disciplines in terms of how dominant research methodology of discipline impacts on attitudes to re-using materials.
  • Socio-cultural context:
    • Design is an inherently social act as teachers are part of both formal (institutional) and informal communities.  These communities can play a mediational role in terms of learning design practice. It was found that while sharing & reuse of designs is an integral part of LD theory, it tends not fall into educators’ conceptualisation of design for learning.  Instead, educators are more likely to share and re-use when designs are recommended by a trusted body (e.g. individuals from communities they are part of).  This is an interesting point and has implications for the way designers and staff developers approach the sharing and re-use of learning designs.
    • There may be tension between innovative practice and institutional strategies.  This raises question of top-down vs bottom-up approach to educational change.  (Are there other change models that work? c.f. Nichols on Viewpoints project.)
    • The social and institutional influences on deployment of a tool within a particular community are at least as important as the functionality and the look and feel of the tool itself.

Taking a slightly different approach Donald et al (2009) in New Zealand describe their HEART (HEaring & Realising Teaching Voice) Project. Donald et al challenge the common assumption that to facilitate reuse in different contexts, designs must describe learning activities or depict key attributes using a systematic form of notation. This project took as its starting point the view that in order to help educators design, researchers/designers need to find a way to help educators connect with their underlying pedagogical beliefs and practices. While not dismissing the potential value of consistent categorisation, their premise is that teacher-designers who develop conscious awareness of how their beliefs about teaching and learning influence their solution to a learning design challenge will be better prepared to consider what existing learning designs might be employed or repurposed to solve that challenge. (p. 181) This approach may be of particular relevance to the theological education sector, in which engagement with issues of the “heart” is everyday practice.

The OULDI project collected extensive range of empirical evidence on design practice over a number of years.  Summaries of the project can be found on the project website and an overview of the findings are summarised in Conole’s 2013 book.   Some of the main findings regarding educators’ design practice are:

  • 5 “themes” for the design process were identified: the process of design, support and guidance, barriers to innovation, representing designs and evaluation mechanisms
  •  a design life-cycle was identified
  • In reviewing the OULDI project, Conole states: “describing design was seen as a difficult and unfamiliar task. It is evident that there are multiple design solutions to any one design problem.  There are also many options for what can be included and it is hard to interpret designs in a consistent way. Finally, any one design representation is only able to capture partial details.” (Conole, 2013, p.108).  The OULDI project found that there is no one perfect tool for design and individuals have different preferences for the design process.
  • Goodyear is also conducting ongoing research into design practice.


Masterman (2013) notes that data from the Phoebe project demonstrates the problem of “achieving a shared understanding between tool and user”, and that there is an inherent tension between the tool being with too simplistic “recipe-style”, or too vague to be useful. Moreover, she notes that during the initial phase of design educators tend to shuffle content and pedagogic approach in a process of rapid iteration, often this phase is achieved using paper and pencil or mind-mapping software which gives the opportunity to move the elements around freely.  The computer is used to record the design, once these elements have been decided.  However, and this is the key point, “The digital tools designed to support pedagogic planning may therefore not be giving support at the time at which it is most needed.  This particularly elusive stage of the design process needs further empirical study for us to understand it more fully and determine how intelligent digital support might be offered.” (p.74)

This may be a significant point.  Perhaps an actual learning design itself is only helpful if the educator understands the underlying theory and pedagogy and the context (this is perhaps why educators like contextualised designs).  Perhaps the “learning design” that is being represented and shared in some of the LD tools and initiatives to date, is not the right part, or it is not necessarily complete.  There may be a previous step or a part of the process that is not yet currently well understood or captured.  This may be related to the early part of design, the creative, intuitive part and the part that is connected with deeply held beliefs and ways of thinking about teaching and learning?

Promising lines for further enquiry appear to include: the HEART project (are there similar projects to this), Masterman’s writing, the patterns literature, particularly Goodyear.  The concept of TPCK may also be relevant.

Conceptually this line of enquiry sits under the “Learning Environment: Characteristics & Values” component of the LDCM, as well as the “Educational Philosophy” component.  Theoretically this appears to connect with the gap between educational practice and theory and the tension between Learning Design and situativity theory.

Some preliminary thoughts on why it might be helpful and relevant to engage with educators’ pedagogical beliefs and values in my research:

  • it appears to be helpful for LD process
  • it engages the heart/person which is helpful for both teaching and formation
  • it may help educators to engage more easily and effectively with unfamiliar approaches and technologies
  • it enables educators to see the big picture and to identify what is really important and to devise or select appropriate methods/tools/approaches to reach these goals
  • it enables deep learning on the part of the educators
  • it offers a means of promoting subjective awareness and COPs among teacher/designers.

Thoughts on Situativity Theory, COPs and Formation

13 11 2013

My recent reading has taken me on an exploration of the theoretical foundations of learning environments, particularly looking at situativity theory and communities of practice.  In a chapter in Jonassen & Land’s 2000 book, Barab & Duffy set COPs in their theoretical context, that is, within constructivism, and within a subset of constructivist theory: situative theory, sometimes called situated cognition.

A cognitive or instructivist approach to learning emphasises the individual learner and views learning as a process of acquiring knowledge, which is understood to consist of symbolic mental representations. That is concepts are viewed as abstract things. Teaching therefore becomes a means of facilitating acquisition or transferring knowledge.

Constructivist approaches emphasise the social nature of cognition, meaning and knowledge. Knowledge is fundamentally situated in practice and teaching is participative. Concepts are not self-contained entities but tools that can only be fully understood through use. Knowing and doing are reciprocal. Knowledge is situated and progressively developed through activity. Knowing is an activity and is always contextualised.

Situative theory is part of the constructivist approach to learning. Situative theories emphasise reciprocal character of interaction which produces meanings about the social world and also produces identities; “individuals are fundamentally constituted through their relations with the world” (Lave 1993, Lemke 1997, Walkerdine 1997, Wenger, 1998). Situative theory suggests that practice should not be conceived of as independent of learning, nor meaning as separate from the practices and contexts within which it was negotiated. This would appear to be consistent with a Christian or biblical view of practice and learning. It could be worth exploring what a Christian response to constructivism, situativity and COPs. The emphasis on identity could be significant for formational learning. (Cf. Lowe & Lowe’s paper on ecology of learning).

Within situative theory (or situated cognition) Barab & Duffy say there are 2 streams:

  1. Psychological perspective which focuses on how to meet specific learning outcomes and design learning environments to support students’ learning in particular areas through situating content in authentic learning activities or “practice fields”.
  2. Anthropological perspective on situativity which focuses on communities and what it means to learn as a function of being part of a community. This focuses on developing an identity as a member of a community.

Barab and Duffy (2005) explore the implications of these two perspectives for designing learning environments. They note that in the psychological perspective on situativity the unit of analysis is the individual learner and the focus is on engaging learner in authentic tasks that are likely to require the use of the concepts or skills desired. Senge calls this ‘practice fields’. E.g.s include PBL, anchored instruction and cognitive apprenticeship. Design principles for practice fields are:

  • Doing domain-related practices i.e. active doing not just listening
  • Ownership of inquiry by student
  • Coaching and modelling of thinking skills
  • Opportunity for reflection
  • Ill-structured or loosely defined dilemmas
  • Work is collaborative and social
  • Learning context needs to be motivating (real issues)

However they also note that practice fields tend to occur in formal learning contexts and so the situated learning is separated from the society/community in which the practices being learned are of value and use. It is practice, not genuine contributions. This, they suggest can lead to the development of negative identities. It also puts the focus on the work of being a student, with a concomitant emphasis on achieving grades. They suggest that educators need to find ways to connect with real communities of practice (COPs) in order to develop a sense of self in their work in society. Similarly, Hung, Chee, Hedberg & Seng (2005) suggest that schooling can lead to problems with identity formation and lack of authenticity (c.f. Harkness). Hung et al posit that schools create cultures of students while COPs create practitioners. Does this imply that if we want practitioners we need COPs?

In the view put forward by Lave (1997) (the anthropological perspective) COPs allow for the situatedness of identities as well as cognitions. COPs are described by Barab and Duffy as situated social practice in which: “There are no clear boundaries between the development of knowledgeable skills and the development of identities; both arise as individuals participate and both become central to the COP”. (p.29). This is a very interesting statement which resonates with what theological educators mean when they talk about formation.

Barab and Duffy suggest that educators need to move towards a learning-as-participating-in-community approach. They provide an example of a “community of teachers” at Indiana university, designed to help pre-service teachers which:

  • Sets up community of teachers and pre-service teachers throughout duration of study. This personalises learning for students and engages them in learning
  • Teachers/students pair up for mentoring/apprenticeship over whole course
  • Ongoing intensive fieldwork (1 day per week)
  • Student led seminars discussing issues on the ground and lessons learned
  • Allows students to collect body of evidence over time of their ability to teach and that they possess qualities of a good teacher.

Barab and Duffy suggest that the key to meaningful learning is finding a way to link the learning experience into society, by giving students a legitimate role or task through community participation or membership. How might this be applied in theological education?

Barab and Duffy argue that COPs are superior to practice fields. But Hung, Chee, Hedberg & Seng (2005) take a slightly different approach and put them on a continuum, finding a valid place for practice fields in a staged and scaffolded process. This would seem to be a more realistic approach.

Hung et al (2005) propose a framework for fostering a community of practice based on 3 models: simulation, participation and co-determined interactions. These 3 models can be placed along a continuum of learning:

Simulation (e.g. PBL,        > Participation (COP)            > Co-determined interactions
project-based learning,
practice fields)

The three models need to be scaffolded, that is, the educator needs to provide a learning structure and framework for the learner to gradually move along the continuum. Scaffolding creates a path but learners need to do the walking.

If this framework is applied to Learning Design approaches we can see that specific learning designs (e.g. a LAMS sequence) tend to be towards the Simulation end of the continuum. Individual learning designs can thus be viewed as one element in the learning process. Hung et al suggest that learners need other (real) elements and to engage in a COP and to move towards authenticity. In this perspective the design of simulation oriented learning environments is complementary to real communities and not a replacement for them. This may be significant for designing for formational learning. Theological educators would agree that genuine formation needs to be a real and authentic process and that ongoing engagement with practitioners is a vital element. Perhaps a scaffolded approach as suggested by Hung et al which includes elements of simulation, COP involvement and aims at eventual co-determined interactions would be a good approach to scaffolding formational learning?

There are several aspects of communities of practice, described by Hung et al that appear to be of relevance to formational learning:

  • COPs recognise the significance of identity formation as integral part of the learning process and provide a means for nurturing this identity formation
  • COPs form ideal situated contexts through which implicit and explicit meanings are appropriated and negotiated by members of the community (p.160).
  • COPs recognise the importance of situated cognition – learners must experience learning in the actual rich situated context. There are e.g’s of this in pre-service teacher education which would form good models for theological education (preservice ministers).

Theological institutions tend to foster strongly collegial environments among their faculty members. In this sense theological faculty, at least those, closely connected to the college at which they teach are experiencing a COP. Faculty members recognise the value of this and so they may well recognise the value of it for students also. It would be worth exploring what other communities of practice currently exist in the real contexts in which theological graduates will be working (e.g. ministers fraternities, mission communities) and to look at how students might be able to become participants in these communities as part of their learning experience.

Hung et al set up a COP for Heads of Departments in Singapore Schools which had an instructivist not constructivist culture. The article describes the successes and challenges of attempting this. The researchers based their approach on ethnomethodology which attempts to consider the contextual nature in which everyday activities are based, focusing particularly on participant categories, i.e. on the way participants organise themselves not what they said about their lives. Ethnomethodology uses conversation analysis, in this case of both f2f sessions and online discussions

The project scaffolded the learning of HODs aiming to make them experience culture and spirit of COPS. This was done in a gradual and staged manner, helping participants to progress from the role of a novice to observer to participant and eventually to active contributor. Might a similar approach be applied to scaffold formational learning in theological institutions, and beyond into the communities of practice relevant to the students ministry?


Sasha A Barab & Thomas M Duffy (2000). “From Practice Fields to COPs”. In David H Jonassen  & Susan M Land, Theoretical Foundations of Learning Environments, Laurence Erlbaum Associates (LEA) Publishers, Mahwah, New Jersey, 2000, pp.25-55

David Hung, Tan Seng Chee, John G Hedberg & Koh Thiam Seng, (2005). “A Framework for Fostering a Community of Practice: Scaffolding Learners through an Evolving Continuum”. British Journal of Educational Technology, 36 (2), 2005, 159-176