LAK19 – Workshop on Advances in Writing Analytics: Mapping the state of the field

Tempe, Arizona, 4 Mar 2019 (Full day)

#WALak19 #LAK19

Call for participation

The fourth Writing Analytics workshop will be of interest to a wide range of LAK delegates including: students and researchers engaged in writing research and the use of writing tools; educators in schools, universities and businesses; data analysts; and companies active or potentially active in the field of writing analytics. Participation will be ‘mixed’ – in addition to participants who present their work, any interested delegate may register to attend. An invitation will be extended to participants of previous workshops, writing researchers who are not (yet) involved with the technology side, and international researchers active in the field to share their work and different perspectives on Writing Analytics. An open call for participation will be put out to encourage others to present their research and become more actively involved in the LAK writing analytics community. The workshop thus welcomes three types of participation: 1) Invited participants to share their work 2) Accepted participants to present their paper 3) Contributors interested in joining the discussion.

To present your work in the workshop, you can choose to submit one of the following (in the companion proceedings template) describing your innovative work in writing analytics and its application in authentic writing scenarios. Accepted presentations will be published in LAK 2019 companion proceedings and linked to the website.

Short paper (5 pages)
Demo/ poster paper (2 pages)
Extended abstract (1 page – for invited participants only)

All submissions should be emailed to Ming.Liu@uts.edu.au by 3 December 2018 with the subject line ‘LAK Writing analytics workshop submission’.

What do we expect?

We are interested in mapping the current field of writing analytics across a wide variety of topics listed below, with a thread of connecting writing analytics to pedagogy. We’d like to hear the kinds of work done in the field that can glean new insights from research to develop theories in writing and how they can be practically implemented for particular contexts. Your paper should include details on the type of writing data, tools used, target writers and evaluation methods in your research. New approaches and prototypes in designing for, analyzing, and visualizing writing data are encouraged to be shared, along with their theoretical and pedagogical underpinnings.

Potential topics include, but not limited to:

Writing tools for students and teachers
Automated feedback on writing
Pedagogical integration of writing analytics
Contextual and institutional drivers for writing analytics
Data collection methods in writing
Analysis methods of writing data
Visualization techniques with writing data
Collaborative and peer-involved writing
Writing research, with potential for computerized support
Theories and frameworks for writing analytics

Important Dates

~~3 Dec 2018~~ 11 Dec 2018 (Extended) – Submission deadline
4 January 2019 – Acceptance notifications sent out
8 January 2019 – Early-bird registration closes for LAK19
5 February 2019 – Camera-ready deadline for accepted papers
4 March 2019 – Writing Analytics Workshop at LAK19, Tempe, Arizona

Workshop focus

Writing Analytics as a field is growing in terms of the tools and technologies developed to support student writing, methods to collect and analyze writing data, and the embedding of tools in pedagogical contexts to make them relevant for learning. This workshop will facilitate discussion on recent writing analytics research by researchers, writing tool developers, theorists and practitioners to map the current state of the field, identify issues and develop future directions for advances in writing analytics.

The proposed fourth workshop in the series will build on the previous writing analytics workshops to develop writing analytics literacy and map the field for the future. The focus will be on critically assessing the current state of work being done in the field, and how it could be directed towards the future by considering key issues. The key thread of integrating writing analytics with pedagogy will be emphasized, by connecting theory, pedagogy and assessment to close the feedback loop (Knight, Shum, & Littleton, 2014; Shibani, Knight, Buckingham Shum, & Ryan, 2017). The pedagogic relevance and the question of why writing analytics is employed and what it can add to the existing system will be brought into discussion by practitioners. In this way, we maintain a productive dialogue among different stakeholders like educators, researchers and developers for effective implementation of learning analytics in the classroom (Thompson et al., 2018).

The landscape of tools that offer support for writing is constantly changing with new tools getting introduced and the existing ones getting updated, to incorporate the technical advances and the data made available over time (Liu, Calvo, Pardo, & Martin, 2015; McDonald, Moskal, Gunn, & Donald, 2018; Rapp & Ott, 2017; Woods et al., 2017). The ways in which we study writing, and respective systems that support its instruction and practice, have also considerably changed with technological affordances like keystroke-level analysis which allow for a more fine-grained level of analysis, and multiple sources of data which allow for triangulation and validation while studying writing processes. It is important to share knowledge from related work on writing, for instance process-mining and temporal analysis, that can contribute to writing analytics research. This will expand the knowledge base of the community and find relevant opportunities to meaningfully collect, analyze, visualize and use data to derive insights that are relevant for the learning contexts. Hence, the workshop will encourage presentations on various tools and techniques to understand and improve writing.

With growth in the field of Writing Analytics, the multidisciplinary of the field, and the different ways in which researchers engage with its development, it is important to align the goals of the field within the community. Community building generates a shared understanding and common goals to work towards the future of the field. While considering the potential pathways for the field to progress, we will also include discussions on the pushbacks and critical perspectives that can affect how the field moves forward. This includes legal and ethical considerations on the use of students’ data, development of learning theories to support writing analytics technology, and evaluation methods to assess these advances for their real impact to meaningfully contribute to writing.

Thus, the fourth workshop is intended to:

Build on the existing dialogue around developing writing analytics literacy and pedagogic integration by connecting different stakeholders like practitioners and researchers.
Expand the knowledge of the field by discussing about novel approaches and tools being developed by different researchers that contribute to writing analytics research.
Move the field forward by building a community for writing analytics research and thinking about pushbacks and potential future steps.

Background

As technological capabilities progress in the field of understanding natural language, there is increasing interest in their application to study and improve writing. Writing analytics has emerged as a sub-domain of learning analytics to support the analysis of written products and processes in educational contexts (Buckingham Shum et al., 2016). The time-consuming and labor-intensive process of assessing writing makes it hard for educators to provide formative feedback on students’ writing, which could be supported by writing analytics. An application of writing analytics that has gained traction is the use of tools that provide automated feedback and writing instruction to improve students’ writing skills (Allen, Jacovina, & McNamara, 2015; Liu, Li, Xu, & Liu, 2017; Woods, Adamson, Miel, & Mayfield, 2017). Such tools developed across different educational levels engage students directly to aid in the improvement of their writing skills. Another objective of writing analytics tools and techniques is to understand the writing products and processes deeper to contribute to the theory and research on writing, which can then lead to its application in writing contexts (McNamara, Graesser, McCarthy, & Cai, 2014). In addition to studying user behavior and interaction through log data, this can inform design choices in writing tool development. These applications build on the main notion of developing a synergy between writing analytics technology and pedagogical practice, so that the educational context is meaningfully embedded in the use of these technologies. Three previous workshops run on this topic have focused on critical perspectives and community building around writing analytics in LAK (Buckingham Shum et al., 2016), developing a writing analytics literacy and practitioner capacity (Knight, Allen, Gibson, McNamara, & Buckingham Shum, 2017) and a hands-on-training for developing this literacy by understanding technical affordances and aligning them to pedagogical feedback (Shibani, Abel, Gibson, & Knight, 2018).

Provisional Programme

The full-day workshop will include a number of presentations and demonstrations from researchers to share their work within the writing analytics community. It will include round-table and open discussions throughout the day to steer the direction of writing analytics work and possible pathways for future advances in the field. Presentations include the following:

Invited talks (15 minutes each):

WAT: Writing Assessment Tool
Danielle S. McNamara 1, Laura K. Allen 2, & Scott Crossley 3
1 Arizona State University, 2 Mississippi State University, 3 Georgia State University
Understanding the ‘Black-Box’ of Automated Analysis of Communicative Goals and Rhetorical Strategies in Academic Discourse
Elena Cotos
Iowa State University
Learning with Text: Toward a Multi-Dimensional Perspective on Text-Based Communication
Laura K. Allen 1, Lacey C. Zachary 1, Danielle S. McNamara 2
1 Mississippi State University, 2 Arizona State University

Short presentations (10 minutes each):

A Novel Writing Analytics Approach to Study Multiple Information Sources Integration by Students
Yoram M Kalman*, Ester Adam, Ina Blau
The Open University of Israel
Am I planning smart? – Analyzing student goals
Sebastian Wollny, Jan Schneider, Marc Rittberger, Hendrik Drachsler
DIPF | Leibniz Institute for Research and Information in Education
Goethe University, Frankfurt am Main
Open University of the Netherlands
Towards knowledge-transforming in writing argumentative essays from multiple sources: A methodological approach
Mladen Raković , Zahia Marzouk, Daniel Chang, Philip H. Winne
Simon Fraser University
Tool Demo: AcaWriter for automated writing feedback
Antonette Shibani 1, Ming Liu 1,2
1 Connected Intelligence Centre, University of Technology Sydney, Australia, 2 School of Computer and Information Science, Southwest University, China
Learning analytics with thesis writer
Christian Rapp
ZHAW, School of Management and Law, Center for Innovative Teaching and Learning, Winterthur, Switzerland

The provisional programme is given below:

Introductions (30 minutes): Introductions of workshop organizers and participants, and a quick background to the field of writing analytics.

Presentations: Presentations and demonstrations from accepted papers and invited researchers on their writing analytics tool or technology, the data collected by the tool, analysis of writing data and how it contributes to writing theory, and the direction of future work.

Q&A, Discussion Blocks: Discussion blocks will follow each presentation to ask critical questions on what can be done and analyzed from the tool/data, how and why.

Round-table discussion: Key topics for discussion from the presentations will be selected for round-table discussion. Participants can move around tables to discuss more in detail on the topic they are interested in.

Lunch Break (1 hour)

Q&A, Discussion Blocks: Discussion blocks will follow each presentation to ask critical questions on what can be done and analyzed from the tool/data, how and why.

Open discussion (30 minutes): Open discussion facilitated among all participants on the advances in writing analytics and its potential future, co-creation of shared notes and resources.

Writing analytics community engagement (30 minutes): Building the community of writing analytics researchers by connecting existing and new researchers in the field. Formation of a formal writing analytics committee if participants are interested.

Concluding remarks and future directions (15 minutes): Brief summary and closing remarks on the workshop with future steps.

Organisers

Shibani Antonette is a doctoral student in Learning Analytics at the Connected Intelligence Centre, University of Technology Sydney. Her research work is on automated feedback from Writing analytics tools and its integration into the classroom for pedagogic use. She uses text analytics for analyzing writing and revision behaviors, and studies the interaction of writers with automated feedback. She has chaired prior Writing Analytics workshops at ALASI and LAK.

Ming Liu is a research fellow of text analytics at the Connected Intelligence Centre, UTS. His research work is focused on researching and developing automated feedback tools that support writing, reading and peer reviewing in the context of individual learning and collaborative learning using learning analytics and artificial intelligence. His research findings have appeared in IEEE Transactions on Learning Technologies, Journal of Internet and Higher Education, Educational Technology & Society and Intelligent Tutoring System.

Christian Rapp is a lecturer and senior researcher at the Centre of Innovative Teaching and Learning at the Zurich University of Applied Sciences’ School of Management and Law. His main research interests lie in the field of developing and implementing teaching innovations combined with complementary research. He has initiated and coordinated several international and interdisciplinary research projects and is currently principal investigator in an EU project further developing “Thesis Writer” (www.thesiswriter.eu), an interactive system supporting writers, supervisors and institutions in managing student dissertations.

Simon Knight is a Lecturer at the Faculty of Transdisciplinary Innovation, University of Technology Sydney. His research focuses on the relationship of analytics to epistemology, pedagogy and assessment, discourse analytics, and epistemic cognition, particularly around information seeking, work which has been presented at LAK and ICLS. He co-chaired the ICLS14 Workshop on Learning Analytics for Learning and Becoming in Practice and LAK15 Workshop on Temporal Analyses of Learning Data. He has also chaired Writing analytics workshops in LAK16, LAK17 and LAK18.

Links to the previous Writing Analytics workshop websites can be found here:

LAK16: Critical Perspectives on Writing Analytics

LAK17: Writing Analytics Literacy – Bridging from Research to Practice

LAK18: Turning the TAP on Writing Analytics