Writing is a fundamental learning activities across learning contexts. With the recent advancement of natural language processing techniques, text analytics are able to identify salient textual features of student essays. However, how to map the textual features to writing feedback could be different in a different learning context. This workshop seeks to connect the technicians and learning designers that work on writing feedback generation. We will demonstrate how (Natural Language Processing) NLP technologies can analyse writing from learning contexts, and how feedback can be created from these computational analyses. Participants will work with a set of text analysis code to extract features using the Text Analytics Pipeline (TAP) and map them to writing feedback using two case studies. Participants with extended technical knowledge will also be given the opportunity to develop rules based on extracted text features to write feedback for their own pedagogic contexts with additional support.

Our tentative schedule is:

• Introduction (15 mins): Introducing the objectives, organizers and technical set-up for participants.
• Introduction to writing analytics and feedback (25 mins): The basics of using Jupyter notebooks and feedback from the Text Analytics Pipeline (TAP). An example of mapping text features to feedback.
• Tutorial and discussion Part I (1 hour): Try AcaWriter (using assignment code(s)). Creating feedback based on features (participants work with sample texts and analytic-features to create rules-based feedback for students).
• Tutorial and discussion Part II (1 hour): Case Study 1: Working with analytic rhetorical moves. Hands-on engagement with text analysis using Jupyter notebooks – extracting features from text and turning them into feedback (using graphQL including the rhetorical features).
• Closing remarks (20 mins): Wrap-up and discussion of next steps, and challenges for the field (shared annotated data, shared text infrastructure and features to build into these tools, and evaluation)

Participants will be asked to work in teams for part of the workshop. We particularly welcome participants from:

• Learning design and discipline expert contexts – if you’ve thought deeply about how to give feedback on writing, get involved!
• Technology and computer science contexts – if you’re familiar with providing analytics (not necessarily text analytics, although that’d be great), come along!
• Participants could consider signing up in teams with a mix of participant roles.
• Participants might like to bring along sample paragraphs of typical academic writing, and more reflective kinds of writing, to test in an analytics system.

The participants will require a laptop or high-end tablet that is running a current

web browser (e.g. Chrome, Firefox, Safari, Edge, or IE10). In addition, the participants are encouraged to bring their student essays.

At the conclusion of the workshop, the participants should have:

• gained some insight into how current NLP technologies can be used to analyse textual data
• developed a basic text analysis process and applied it to an example text improved their understanding on how to apply text analytics in writing through a co-design feedback rule generation approach with educators
• increased their knowledge of Learning Analytics with textual data

Ming Liu is a research fellow of text analytics at the Connected Intelligence Centre, UTS. His research work is focused on researching and developing automated feedback tools that support writing, reading and peer reviewing in the context of individual learning and collaborative learning using learning analytics and artificial intelligence. His research findings have appeared in IEEE Transactions on Learning Technologies, Journal of Internet and Higher Education, Educational Technology&Society and Intelligent Tutoring System.

Simon Knight is a lecturer at the Faculty of Transdisciplinary Innovation, UTS. A strand of his research is on the design of learning analytics, with a particular focus on writing analytics. He also researchers how people evaluate and use evidence, particularly quantitative data, including educator’s use of evidence in their learning design. He is an editor of the Journal of Learning Analytics.

Shibani Antonette is a Doctoral student at the Connected Intelligence Centre, UTS. Her research work is focused on developing writing analytics tools that are integrated in the classroom for pedagogic use. She uses text analytics for analysing student essays to study their writing and revision behaviours, and for developing features that can be used to provide automated feedback on student drafts.

Sophie Abel is a Doctoral student at the Connected Intelligence Centre, UTS. Her research is focused on how writing analytic tools can be used to help Higher Degree Research (HDR) students improve their research writing. She is particularly interested in how ‘good’ features of academic writing can be instantiated as text analytic features in writing analytic tools.