Date/Time
Date(s) - 22/11/2022
2:10 pm - 4:10 pm
With the rise of extremely large language models, the field of Natural Language Processing (NLP) is increasingly asking which rules of grammar and abstract linguistic classes they have really learned. Do they have a similar understanding of language as humans? Within human language processing, structural information is assumed to be an integral part of our internal representations. It is therefore an interesting endeavour to approach the interpretation of language models from a similar perspective. As such, language models become psycholinguistic subjects themselves. By designing controlled behavioural experiments we can better understand how different contexts affect their behaviour.
This talk will present recent work wherein we investigate the extent to which modern neural language models are susceptible to structural priming, the phenomenon whereby the structure of a sentence makes the same structure more probable in a follow-up sentence. We explore how priming can be used to study the potential of these models to learn abstract structural information, which is a prerequisite for good performance on tasks that require natural language understanding skills. We find that Transformer models indeed show evidence of structural priming, but also that the generalizations they learned are to some extent modulated by semantic information. Our experiments also show that the representations acquired by the models may not only encode abstract sequential structure but involve a certain level of hierarchical syntactic information. More generally, our study shows that the priming paradigm is a useful, additional tool for gaining insights into the capacities of language models and opens the door to future priming-based investigations that probe the model’s internal states. Even more generally, they illustrate the benefits of repurposing experimental paradigms from psycholinguistics to investigate the knowledge acquired by large language models.
Time and location:
22 November at 14:10.
Zoom/Edinburgh George Square; link/specific location will be shared with registrants.
Registration:
Dr. Sinclair’s talk will be hybrid; please register at https://edinburghinformatics.eu.qualtrics.com/jfe/form/SV_2laDCqDTQTyherY. Attendees wishing to meet Dr. Sinclair in Edinburgh may contact her directly at arabella.sinclair@abdn.ac.uk.
