Syntactic Complexity of Learning Content in Italian for COVID-19 Frontline Responders: A Study on WHO’s Emergency Learning Platform

1. The complexity of the knowledge content on health emergency learning platforms: focus on a syntactic strategy

The recent COVID-19 pandemic has increased “the demand for trusted information to help frontline personnel and communities respond to the outbreak” (Gamhewage et al. 2020, p. 257). While it is financially insignificant to replicate and disseminate information (David & Foray 2002), it takes much more effort to reproduce knowledge through learning (Foray 2001). To be specific, effective knowledge reproduction requires information accessibility of the learners, intended as a computational cost in retrieving content. Characteristics such as simplicity, exhaustiveness and computational costs for the addressees, or users in the context of online learning, are keys to reproducing actionable knowledge.

Linguistic barriers have been observed reducing the accessibility of users, thus halting the knowledge transfer process, given that knowledge is created and reproduced through human language. The complexity of the knowledge content often determines the required efforts and potential challenges during the language translation process. This is particularly true in the context of health emergencies in which the medical information needs to be accessed by the affected population where a local substandard variety is spoken (Translators without borders, 2017). Being able to evaluate the linguistic complexity of the source content and the linguistic variability of the target population could potentially facilitate the knowledge transfer process during a public health or societal emergency event.

An accessible domain of linguistics for quantifiable variables in corpus-based studies, keeping a crosslinguistic dimension, is the domain of syntax. What formal linguistics can do is to provide a definition of a quantifiable computational (syntactic) complexity, measured in terms of complex syntactic structures and operations (Cinque & Rizzi 2016 for an overview), to be investigated in various genres targeting different addressees.

Are we able to quantify such dimensions of variation? What would be possible measurements to linguistically quantify the levels of complexity in the health learning contents?

The paper aims to explore the above-mentioned questions by investigating the level of complexity of the linguistic content extracted from the open learning platform OpenWHO hosted by World Health Organization (Rohloff et al. 2018; Zhao et al. 2019). It serves also as a case study of assessing computational complexity of a specific syntactic construction in Italian language, represented by the frequency of a typology of relative clauses.

To accomplish our goals, we first analyse the nature of the computational ranking costs of a typology of relativization strategies, narrowing the analysis on Italian in order to provide both qualitative and quantitative insights.

In this work, we quantify a single case of syntactic complexity discussed in results in psycholinguistics and cognitive studies (Friedmann et al. 2009). We focus on a set of structures that are proven successful to show clear asymmetries in terms of parsing in studies of language development and language pathology (Martini et al. 2019, for an overview). We adopt the computational point of view stating that “differentials in frequency are hallmarks of underlying grammatical properties” (Merlo 2016; Samo & Merlo 2019, p. 3), and postulate that the more complex a syntactic configuration in terms of cognitive impact is, the less frequent would occur in a specific dataset.

We first determine a quantifiable notion of complexity by selecting one specific type of constructions as a grammatical (in terms of accepted and naturally occurring) “complex” structure, which is the relativization by embedding on the subject (e.g., “the authors that write this sentence are linguists”) and objects (e.g., “this sentence that the authors are writing is an example”).

We then determine a quantifiable dimension of complexity, which is represented by the frequency of relative clauses, both on subjects and objects, in restrictive and non-restrictive cases. Relative clauses convey different two layers of complexity. A first level is associated with the fact that in every relative clause, one of the components (the relativizer) logically represents a variable that needs to be parsed and indexed. Deliberately keeping technicalities that are beyond the scope of this paper, studies in psycholinguistics (Frauenfelder et al. 1980) have shown that subject relatives are computationally easier to parse than object relatives. Object relatives, though grammatical for adult speakers, are parsed slower and result partially ungrammatical in early grammars (Friedmann et al. 2009) and in specific language impairments and atypical development populations (on aphasia, Grillo 2008; on Autism Spectrum Disorder, Durrleman et al. 2015). In a similar vein, these structures challenge neural-network-based techniques in Natural Language Processing (Merlo & Ackermann 2018). The theoretical account developed in Rizzi (1990, 2013) states that this is due to the fact that the subject represents an intervener for the extraction process from the locus of generation of the object (the “right” of the verb in a Subject-Verb-Object language) to the very beginning of the relative clause (which displays an Object-Subject-Verb order). On the other hand, subject relatives are better mastered since no intervention is at play (Friedmann et al. 2009).

In this paper, we conduct a study on Italian. In Italian (as previously given in the English examples), object relative clauses involve the displacement of syntactic constituents. The peculiarity of Italian is that the construction of object relatives modifies the canonical order (Subject-Verb-Object) in three possible patterns: (i) Object-relativizer-Subject-Verb (la frase che l’autore sta scrivendo ‘the sentence that the author is writing’), (ii) Object-relativizer-Verb-Subject (la frase che sta scrivendo l’autore, displaying a post-verbal subject), or (iii) Object-relativizer-Verb (la frase che sta scrivendo, with a null subject). In all the cases, we observe a reordering of the canonical syntactic architecture of Italian.

Our linking hypothesis is to formulate a strictly mutual interaction between the complexity of a structure and the frequency of complex structures in a specific corpus extracted from the OpenWHO emergency learning platform, whose content addresses COVID-19 frontline responders.

In this work, we analyse syntactic complexity at both an intratextual level and a cross-textual level. Two research questions are investigated.

The first research question focuses on the frequencies of subject and object relatives in the emergency platform’s content. We expect that more complex structures (object relatives) should result in fewer occurrences with respect to less complex configurations (subject relatives). This hypothesis can be stated in H₁.

H₁: The raw counts of subject relatives are expected to be greater than the raw counts of subject relatives in the content of an emergency platform.

As for the cross-textual analysis, a comparison with other text genre conveying information (e.g., news, encyclopedic entries, social media) is required. The second research question targets the typology of text genre. If compared with other types of genres, a corpus for health emergencies should ideally reduce complexity. To do so, we build a simple computational model of expected and observed counts. We can state the second hypothesis in H₂.

H₂: The textual content extracted from OpenWHO should be less complex than other genres (e.g., news, encyclopedic entries, social media).

In section 2, we provide a description of the materials and methods used for the study. In section 3, we restate our hypothesis according to the presented materials and methods. In section 4, we demonstrate how the syntactic complexity found in the OpenWHO corpus in Italian is reduced and differs with respect to corpora conveying information content via news, encyclopedic entries and social media. Finally, section 5 concludes.

2. Materials & Methods

3. Quantifying Hypotheses

The asymmetry between subject relatives and object relatives should quantitatively result in different frequencies. Therefore, we expect that subject relatives should occur at a higher frequency than non-subject relatives. According to our materials, we can therefore restate H₁.

H₁: The raw counts of subject relatives are expected to be greater than the raw counts of object relatives in all the corpora under investigation. A ratio given by the number of the object relatives divided by the number of subject relatives should be minimized for the who corpus.

As it concerns the second hypothesis, the predictions are formulated in terms of differences in numerical observations. In this case, we study the difference between the expected counts and the observed counts. We expect that a quantitative analysis on the who corpus shall show a reduced complexity compared to the other treebanks of texts when delivering knowledge. We thus expect to quantify these results in detecting a smaller frequency of the relative clauses (for both subject and object-oriented relative clauses) in the who corpus with respect to other treebanks. We can thus restate H₂.

H₂: The coefficient of the complex structures (D^WHO/PUD, D^{WHO/ ISDT}, D^{WHO/ TWIT}) for both subject and object relatives should be negative.

Both hypothesis H₁ and H₂ are to be contrasted to a null hypothesis that would predict that the nature of a knowledge transfer platform does not correlate to the imputed values of the relation expected/observed counts.

4. Results and discussion

Table 1 confirms H₁ since the raw counts of subject relatives are greater than the raw counts of object relatives in all the corpora under investigation. Focusing on the asymmetry between subject and object relative clauses, we confirm the observation that different corpora show very large fluctuations in distributions of grammatical relative clauses (confirming trends observed in Roland et al., 2007), as it can be observed by the proportions of subject and object relatives in Table 1. A more important piece of evidence lies in the ratio between subject and object relatives. Independent of the raw numbers of relative clauses, the ratio is extremely similar in all the three “control groups” (pud 0.58, isdt 0.59, twit 0.62). As predicted, this ratio drops dramatically in the corpus who (0.04), indicating a reduction of complexity in the specific corpus. The findings confirm the predictions that complex structures as object relatives are avoided (if necessary) in the context of learning in emergencies.

Table-1: Distribution of relatives

Source: Authors Calculation. Subjects and Objects Relatives in the treebanks under investigation.

Table 2 confirms H₂. All the Ds under investigation are negative. While the reduction of subject relatives varies among the corpora, the decrease of expected object relatives shows a percentage higher than 93%.^[7]

Table-2: Observed vs. expected counts.

Source: Authors Calculation. Observed counts, expected counts, D coefficients and binomial test for subject and object relatives. Results confirming the hypothesis are given in bold.

Both hypotheses are therefore confirmed. We hereby conclude that the emergency platform presented in this study reduces its ‘syntactic’ complexity to possibly transfer knowledge in a fast and efficient way.

5. Conclusion

To sum up, reduced computational costs in terms of syntactic complexity (focusing on relative clauses) are clearly observed in the source data studied in this paper, compared with those of corpora conveying information in other genres (news, encyclopedic entries, social media material, etc.). Moreover, theoretical syntax offers a quantifiable measure to detect layers of complexity in texts, which possibly leads to an improved speed of knowledge transfer processes in emergency contexts.

The simple computational model presented in this paper contributes to providing a relatively simple methodology for detecting layers of complexity in specific genres such as learning platforms. We envision potential applications in adopting syntactic complexity as a measure in classification tasks.

In future works, the inclusion of a multilingual dimension could be considered to research on syntactic strategies and observe whether such a syntactic complexity is actually reduced crosslinguistically. A comparison with a variety of health emergencies or types of corpora made for medical guidelines would also be valuable.

The findings discussed in this paper explore the important role of how corpus studies could potentially play in the knowledge transfer process for a specialised domain such as public health. The conclusion that a digital learning platform such as OpenWHO delivers the maximum amount of information in a less complex nature of language. Last but not least, this study offers opportunity of further cross linguistic analysis in the domain of formal linguistics and provides an entry point for learning experts to determine the required digital learning standard in the context of multilingualism.