Summary of Research in Learning Design

27 11 2013

Summary of Research in Learning Design

This is by no means a comprehensive summary but more of an in-progress, first impressions summary of research developments in the field of learning design, which I will add to as I read and discover more.

Aims (aspirations) of Learning Designers & Learning Design Research (LDR):

  • Core concepts of LD are: guidance, representation and sharing (LDCM)
  • Making design processes more explicit and shareable (Conole, 2013)
  • Creating more effective learning environments and interactions for learners with the aim of assisting effective and meaningful learning.
  • New approaches are needed to help educators make effective use of technologies + educators often not experienced in design.
  • To help educators move from practical consciousness (we do it this way) to discursive consciousness (I am doing this because) based on Giddens, from OULDI-JISC report. (Note this is what HEART project is aiming to do).
  • “development of tools, design methods and approaches to help teachers design pedagogically effective learning activities and whole curriculum which make effective use of technologies.” (Conole, 2013, p.6.)
  • One aim is to generalise across cases, streamlining the process of future design by offering general principles of application, or even universal patterns (Beetham & Sharpe 2007).
  • 2 goals (Bennett et al 2009): to provide a means for teachers to document and work with their designs for planning and implementation; and to provide a way of sharing, adapting and reusing designs. “The aim is not to prescribe a particular design to be copied but to extend a teacher’s pedagogical repertoire through the process of modifying someone else’s design.” P.155

Overview of Learning Design Research
It appears that within the field of Learning Design there was an initial “sprouting” of a variety of different approaches, initiatives and tools which aimed to find ways to represent and share learning designs (e.g. IMS LD, LAMS, LDSV, Phoebe, London Pedagogical Planner/Learning Designer, Pedagogical Patterns Collector, Compendium LD.  See Larnaca Dec website for a summary and timeline of these).  Agostinho et al (2008) categorised these “LD languages” into 4 categories:

  • Pedagogical patterns (Phoebe)
  • Generic lds i.e. patterns & generic ld visualisation sequences (LDVS)
  • Contextualised ld instantiations (LDVS, LD Lite and E2ML)
  • Executable, runnable versions (IMS LD, LAMS)

Some of these sprouts withered and died and others have survived and over time there was a discovery by researchers that practitioners don’t necessarily use the learning designs (either at all, or in the way intended).  This limited uptake and sharing of learning designs ‘not invented here’ (Philip& Cameron, 2008) then led to further research into why this might be the case.

One strand of research looked at whether a LD can be consistently and clearly represented.

A literature review of international research from 2004-2008 by Agostinho et al 2009 found that effective lds share three elements:

  • clear and explicit description of pedagogy of ld
  • some form of quality rating of the ld (eg evaluative findings)
  • explicit guidance/advice about how the ld could be used

Agosthinho et al (Ch 6 in Beetham & Sharpe, 2013) also identified a need for embedding design support in LMSs but noted technical difficulties in implementing this.  These authors are now planning to carry out further research into lds as a stimulus for design thinking (using case-based reasoning).

Masterman (2013) also addresses this issue and concludes that there are 3 ongoing issues for representing learning designs:

i. providing computational support for cognitive actions in an “ill-defined” domain
Issues include: how to design guidance for educators that is useful without being overly constraining, not too simplistic, nor too vague/general; the elusive nature of the early stages of the design process when many things are being shuffled and managed in rapid iteration (often with paper/pen or mind-mapping tools) – this is difficult to capture.
ii. Positioning the tools vis-à-vis theory
Given the multiplicity of theoretical influences on teachers Masterman suggests a supportive digital tool that is rooted in “a principled theory of educational processes” is needed rather than tool that is built on a prescriptive framework. (What does she mean by this?)
iii. Politics of deployment
Masterman identifies several issues in the deployment or implementation of lds including: tensions identified in design itself (method vs creativity); theory-informed vs pragmatic approaches; the ill-defined nature of LD vs systematisation into an ontology to underpin computational support; conformity vs creativity in educational practice.

Conole (2013) refers to Gibbons & Brewer’s (2005) summary of the dimensions of design languages which provides a good summary of the challenges inherent in representing design:

  • complexity – design is a partial representation of a complex multifaceted ideas
  • precision – tension between natural, fuzzy, real practice and highly defined specification
  • formality & standardisation – do the terms used mean the same thing to all users
  • tension between personally created designs and those shared with others
  • designs are dynamic, not static artefacts & co-constructed in context
  • issues around computability.

Falconer & Littlejohn (2008) identified an overlapping set of challenges for LD representation:

  • Ownership of lds
  • Representations need to fulfil multiple purposes
  • Generic vs detailed i.e. used for orchestration or inspiration
  • Designs are both product and process
  • Degree of granularity

The recent development of the LDCM has provided a conceptual framework in which to place these issues though it has not necessarily solved them.  Challenges such as the degree of granularity can be relatively easily managed by careful and consistent use of language.  Other challenges such as the inherent complexity of design, the fuzziness of real practice,  need for representations to fulfil multiple purposes and whether they are used for orchestration or implementation are in reality elements of tension in Learning Design research and practice, which researchers and practitioners need to be aware of.  As the field continues to develop and mature researchers and practitioners are likely to find ways of managing these tensions within research and practice. Ongoing research and development of how to represent learning designs is continuing in multiple contexts. The field as a whole continues to work towards developing a widely acceptable descriptive framework for l & t activities (or “educational notation”.(Larnaca declaration, p.5)  Though, as the Larnaca Declaration notes “If a notation system (or systems) for describing teaching and learning activities is developed and widely adopted, its success will be due to a complex mixture of its accuracy, expressiveness and historical contingencies.” (p.31)

A second and connected strand of research has focused on exploring educators’ actual design practice in more depth to work out whether any insights could be gained from authentic design practice that could then feed back into learning design models and approaches.  This strand overlaps with the first and often involves the same researchers but the focus is more on the guidance and sharing aspects of LD rather than its specific representation.  This strand explores questions and issues around how to help educators to describe, share, adapt and reuse lds.  It is about designing for learning (Beetham & Sharpe, 2013) or Learning Design Practice.

Agostinho, (Ch 7 in Beetham & Sharpe, 2013), found that:

  • lds (using LDVS representation method) could be readily understood and used
  • lds are used for design ideas ie as a source of ideas rather than a model to replicate
  • lds are used for benchmarking i.e. teachers use to assess their previous and current work and build confidence. In this sense lds are models of good practice not descriptive pedagogy.
  • designing with a ld led to an improvement in teachers pedagogical content knowledge (PCK, Shulman 1980) and TCPK (Mishra & Koehler, 2006).  TCPK follows (or perhaps is secondary) to PCK.
  • teachers prefer contextualised examples (where the design is described in its original context) rather than a more generic guide. Contextual detail appears to add to useability.  This finding raises questions concerning the usefulness of more generic designs.

Another study by Agostinho in 2011 found that the LDVS (from the 2003 AUTC project) was used to document teaching ideas, as a design tool to discuss teaching ideas with colleagues, or as an analysis tool to reflect on learning design.

Oliver et al (Ch  6 in Beetham & Sharpe 2013) found that the LDVS framework with tasks, resources and supports provides an efficient means to describe learning experience and to represent teachers plans but noted issues with:

  • Teachers come up with different ways of representing designs, even when using the same framework.
  • Difficulty identifying the critical elements – teachers describe things in different ways.
  • Varying size and scope of LDs, which makes comparison difficult.

Some researchers’ attention has turned to finding effective means to portray and communicate the pedagogical frameworks and relevant contextual information to support the reuse of learning designs (Bennett et al., 2007; Philip & Cameron, 2008, in Donald et al 2009).

Masterman (Ch 4 in Beetham & Sharpe, 2013) concludes that the design process is complex, related to individual dispositions of practitioner and a range of intellectual and socio-cultural influences and the nature of the process itself. She suggests that to provide design support and tools we need to understand teacher’s design practice and setting, particularly:

  • Conceptualisation of and approach to activity of design:
    • Design is both systematic and creative, structure emerges from fluidity and negotiation
    • Tension/distinction between planning (laying out of constraints) and design (what can be achieved within constraints)
    • Design is a messy, juggling process – teachers start at different places, take different route and employ different metaphors, modules and visualisations
  • Factors that bear on teacher’s approach to design
    • These include style/approach of teaching they experienced, institutional polices and structures, student related factors, theory, research-informed thinking, nature of the discipline.
    • Theory can both inform the activity of creating a ld and fulfil an explanatory function in the activity of reviewing (reflection).
    • Practitioners tend to draw on number of theories, some take a-theoretical stance. Pragmatism often wins out.
    • There is an increasing awareness of the value of evidence based practice to design.
    • There are differences between disciplines in terms of how dominant research methodology of discipline impacts on attitudes to re-using materials.
  • Socio-cultural context:
    • Design is an inherently social act as teachers are part of both formal (institutional) and informal communities.  These communities can play a mediational role in terms of learning design practice. It was found that while sharing & reuse of designs is an integral part of LD theory, it tends not fall into educators’ conceptualisation of design for learning.  Instead, educators are more likely to share and re-use when designs are recommended by a trusted body (e.g. individuals from communities they are part of).  This is an interesting point and has implications for the way designers and staff developers approach the sharing and re-use of learning designs.
    • There may be tension between innovative practice and institutional strategies.  This raises question of top-down vs bottom-up approach to educational change.  (Are there other change models that work? c.f. Nichols on Viewpoints project.)
    • The social and institutional influences on deployment of a tool within a particular community are at least as important as the functionality and the look and feel of the tool itself.

Taking a slightly different approach Donald et al (2009) in New Zealand describe their HEART (HEaring & Realising Teaching Voice) Project. Donald et al challenge the common assumption that to facilitate reuse in different contexts, designs must describe learning activities or depict key attributes using a systematic form of notation. This project took as its starting point the view that in order to help educators design, researchers/designers need to find a way to help educators connect with their underlying pedagogical beliefs and practices. While not dismissing the potential value of consistent categorisation, their premise is that teacher-designers who develop conscious awareness of how their beliefs about teaching and learning influence their solution to a learning design challenge will be better prepared to consider what existing learning designs might be employed or repurposed to solve that challenge. (p. 181) This approach may be of particular relevance to the theological education sector, in which engagement with issues of the “heart” is everyday practice.

The OULDI project collected extensive range of empirical evidence on design practice over a number of years.  Summaries of the project can be found on the project website and an overview of the findings are summarised in Conole’s 2013 book.   Some of the main findings regarding educators’ design practice are:

  • 5 “themes” for the design process were identified: the process of design, support and guidance, barriers to innovation, representing designs and evaluation mechanisms
  •  a design life-cycle was identified
  • In reviewing the OULDI project, Conole states: “describing design was seen as a difficult and unfamiliar task. It is evident that there are multiple design solutions to any one design problem.  There are also many options for what can be included and it is hard to interpret designs in a consistent way. Finally, any one design representation is only able to capture partial details.” (Conole, 2013, p.108).  The OULDI project found that there is no one perfect tool for design and individuals have different preferences for the design process.
  • Goodyear is also conducting ongoing research into design practice.


Masterman (2013) notes that data from the Phoebe project demonstrates the problem of “achieving a shared understanding between tool and user”, and that there is an inherent tension between the tool being with too simplistic “recipe-style”, or too vague to be useful. Moreover, she notes that during the initial phase of design educators tend to shuffle content and pedagogic approach in a process of rapid iteration, often this phase is achieved using paper and pencil or mind-mapping software which gives the opportunity to move the elements around freely.  The computer is used to record the design, once these elements have been decided.  However, and this is the key point, “The digital tools designed to support pedagogic planning may therefore not be giving support at the time at which it is most needed.  This particularly elusive stage of the design process needs further empirical study for us to understand it more fully and determine how intelligent digital support might be offered.” (p.74)

This may be a significant point.  Perhaps an actual learning design itself is only helpful if the educator understands the underlying theory and pedagogy and the context (this is perhaps why educators like contextualised designs).  Perhaps the “learning design” that is being represented and shared in some of the LD tools and initiatives to date, is not the right part, or it is not necessarily complete.  There may be a previous step or a part of the process that is not yet currently well understood or captured.  This may be related to the early part of design, the creative, intuitive part and the part that is connected with deeply held beliefs and ways of thinking about teaching and learning?

Promising lines for further enquiry appear to include: the HEART project (are there similar projects to this), Masterman’s writing, the patterns literature, particularly Goodyear.  The concept of TPCK may also be relevant.

Conceptually this line of enquiry sits under the “Learning Environment: Characteristics & Values” component of the LDCM, as well as the “Educational Philosophy” component.  Theoretically this appears to connect with the gap between educational practice and theory and the tension between Learning Design and situativity theory.

Some preliminary thoughts on why it might be helpful and relevant to engage with educators’ pedagogical beliefs and values in my research:

  • it appears to be helpful for LD process
  • it engages the heart/person which is helpful for both teaching and formation
  • it may help educators to engage more easily and effectively with unfamiliar approaches and technologies
  • it enables educators to see the big picture and to identify what is really important and to devise or select appropriate methods/tools/approaches to reach these goals
  • it enables deep learning on the part of the educators
  • it offers a means of promoting subjective awareness and COPs among teacher/designers.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: