Making a case for more feminist approaches in quantitative research. How commonly used quantitative approaches in adult education research marginalise and oversimplify diverse and intersectional populations

In contrast to qualitative and theoretical approaches, the mainstream of quantitative research often still finds it difficult to incorporate modern concepts of diversity and intersectionality into its work. This article aims to highlight various aspects in which large studies and their evaluations marginalise or ignore certain parts of the population. In surveying data, large-scale surveys like the Programme for the International Assessment of Adult Competencies (PIAAC) often not only operate on a binary gender concept but also do not differentiate between a person gender identity and their social gender. In addition, commonly used methods keep unequal distributions invisible. Non-binary people are virtually invisible, unequal benefits for women remain hidden and the intersectional diversity inside the broad gender categories poses challenges to the mainstream of quantitative research in adult education. Therefore, there is a need for a feminist approach to statistical methods and quantitative research and in particular a feminist approach to a careful and critical interpretation.


Introduction
From city planning to everyday working life to a more and more digitalised world, many elements of this world have been structured and implemented by the people in power and therefore were and are built to reflect their perspectives and to cater to their needs (Criado-Perez, 2019).Alas, adult education research has been no exception to this rule.In many fields of academia and live, mechanisms take effect that prioritise white middle-class men and their perspectives (c.f.Buvinic, Furst-Nichols, & Koolwal, 2014;Buvinic & Levine, 2016).Following Foucault and his concept of bio-power statistics themselves can be described as a part of neoliberal power mechanisms that hold great influence on societal structures and its subjects (Foucault, 2019).Women and other people who are not only under-represented amongst researchers, but also are their interests and needs marginalised in every step of the research process.While some progress has been made, especially in qualitative research, quantitative research still struggling to incorporate gender-sensitive or inclusive approaches (Else-Quest & Hyde, 2016, p. 319).
In this paper, I will highlight a few steps and aspects in which quantitative adult education research perspectives are gendered and would benefit from a broader variety of voices and approaches.Therefore, this paper will look more closely into two central aspects of quantitative research: The way we accumulate and survey data and how we subsequently handle and analyse the data.To demonstrate these steps, I will utilise the public use files of the Programme for the International Assessment of Adult Competencies (PIAAC).It is an international large-scale assessment, testing literacy, numeracy and technical problem-solving skills of adults (between the ages of 16 and 65) of the resident populations in 38 countries.It is conceptualised and conducted by the Organisation for Economic Co-operation and Development (OECD) and therefore focusses not only on the countries that are members of the OECD but also on their concepts and interpretations of adults competencies and their socio-cultural relevance (c.f.GESIS, 2020;OECD, 2013).While the survey offers broad and extensive data on the situation and circumstances of adult education in multiple countries, it has also been criticised for its singular Western and economic perspective (e.g.Addey, 2018;Duckworth & Smith, 2019, pp. 27f.;Allatt & Tett, 2019, pp. 41f.;Grotlüschen & Heilmann, 2021).
Other referenced large-scale assessments in adult education will be the European Adult Education Survey (AES; c.f.European Commission, 2013) and the German national survey LEO 2018 -Living with Low Literacy (LEO; Grotlüschen, Buddeberg, Dutz, Heilmann, & Stammer, 2020).
All are large-scale assessments that claim representativeness for their respective groups; PIAAC as well as LEO include a competence test as well as a comprehensive (background) questionnaire (Grotlüschen & Buddeberg, 2020;OECD, 2013b) and both are viewed as relevant quantitative research respectively in international discourse and in German adult basic education debates (e.g.Hoogland, Heinsmann, & Drijvers, 2019).
In this paper, I will look at these surveys and their analyses as representatives for mainstream approaches in quantitative research.After reasoning why visibility and representation in research are relevant for gender equality, this paper presents selected elements and aims to demonstrate aspects where gender in general and perspectives of the non-powerful are made invisible in common quantitative approaches.Many of the issues that will be raised in this paper have been voiced multiple times.They are often either used to point towards qualitative or mixed-method approaches, which have traditionally included more critical and feminist perspectives (Westmarland, 2001) or to push for alternative approaches towards quantitative methods and their interpretations.This paper follows the second line of argument and tries to reason and to demonstrate how well known criticism of supposedly objective approaches does apply to the field of adult education.

The necessity of visibility and representation
From different perspectives, the inclusion of women (and increasingly non-binary persons) in the scientific world seem relevant and necessary to further advance the gender equality.This includes diverse perspectives in the research teams and as the objects of research.

Diverse representation in research
Women are continuously described as underrepresented in sciences (Rossi, 1965;Sarseke, 2018).This might indicate mechanisms of exclusion that prevent women from pursuing careers in science.Sarseke (2018) finds 'that the subject 'gender and science' has been looked at for at least three decades, and the results obtained have not changed significantly.' (Sarseke, 2018, p. 98).An image of a 'leaky pipeline' has been used to illustrate the process of women and non-binary people slowly but consistently leaving certain professions or career trails (Buckles, 2019;Pell, 1996).While these effects are more visible in the STEM fields, the generally more diverse fields of adult education show similar distributions when it comes to statistical or quantitative research.
At the same time, women and non-binary people are not homogenous but rather highly diverse groups.The highly different experiences of people of different social classes, of racialized women and non-binary people, and of (non-)disabled individuals cannot be represented by a single female perspective (hooks, 1982;Merrill, 2005).
Not only do women face fundamental disadvantages in many areas of science, this unequal representation also has an impact on the questions asked and the methods used.Homogeneous scientific perspectives can lead to one-sided research questions and approaches, which lead to further stereotyping, discrimination or invisibility of population groups that were already hardly visible or marginalised.

Intersectionality in quantitative research
Following Crenshaw (2017Crenshaw ( ), hooks (1982)), and many others, the concept of intersectionality describes how the highly diverse and complex nature of different group memberships and forms of discriminations cannot be understood by looking at them separately.There is no consensus on what the terms "feminism or intersectionality" mean and they are defined in different ways for different research approaches (e.g.Bührmann, 2010;Degele & Winker, 2007).However, there can be found similarities and a common core of convictions, such as a fundamental belief in an equality of all people disregarding gender, class, language, the colour of their skin, their skills and abilities, and other characteristics.
Scott and Siltanen (2017) looked at common quantitative research methods and questioned how compatible they were with intersectional theoretical approaches.They found that the more complex the view of the diversity and intersectionality of the examined group was, the more inadequate the usual applied methods became (Scott & Siltanen, 2017, p. 374).Combining an intersectional approach with quantitative methods poses a major challenge to researchers.
[T]he methodological choices at our disposal […] are severely limited.Try as we might, it is virtually impossible to escape the additive assumption implicit in the questions we use to measure intersectionality and in our analysis of the phenomenon.(Bowleg, 2008, p. 322)

Surveying data on gender
In order to claim any correlation of any variable to gender, the assessment of a gender variable and the underlying construct is vital.While including a gender variable almost seems to be an automatism: In almost all assessments, gender is surveyed even when gender-related differences are neither part of the research question nor part of the theoretical framework (Magliozzi, Saperstein, & Westbrook, 2016).Instead, gender is often included as a standard variable, which is included anyway and without further thought to a theoretical basis or conceptualisation.

Social gender, sex and gender identity in large-scale assessments
When gender is seen as a complex social construct and gender identity as a non-visible trait of a person, one might take issue with the way gender is surveyed.More often than not, large-scale assessments do not ask for respondent's gender identity but instead ask the interviewer to assume and prescribe a social gender.For example, PIAAC's background questionnaire specifies, 'this question will be recorded by the interviewer through observation […] and only asked of the respondent if needed.' (OECD, 2010, p. 7).The interviewer instruction therefore reads 'Ask only if uncertain.'(OECD, 2010, p. 7) and allows for two valid responses: Male and Female.Similarly, also the AES and LEO left it to the interviewer to determine the participants gender (Eurostat, 2012, p. 2; Grotlüschen, Buddeberg, Dutz, Heilmann, & Stammer, 2019, p. 7).
This reveals a very restrictive view on gender and gender identity.Which is not reflected in more recent understandings of gender identity.Gender is instead viewed as binary (male/female) and as "readable", i.e. as a personal trait that is easily visible.Another person is expected to recognise somebody's gender "through observation" with the expectation of being "certain" in one's attribution most of the time.Besides the theoretical changes and critique, at least since the 1980s efforts have been made to point 'to the differences between personal perceptions of sexual identity and scientific evaluations by objective outsiders' (Stern, Barak, & Gould, 1987, p. 504), meaning that the inaccuracy of surveying gender this way is measurable (Stern et al., 1987).
Attempts to resolve this difference between the theoretical approach and the method of data collection are often met with doubts and hurdles.
For example, there are concerns that in many surveys, expanded categorical measures will yield some populations that would be too small for statistical analysis.Improved categorical measures also do not allow for variation within gender categories; such questions continue to treat gender as a set of discrete attributes, each assumed to describe a relatively homogenous population.(Magliozzi et al., 2016) During the 1970s and 80s, several attempts were made, to survey gender in different scales, for example by asking participants to indicate where they fell on a male/female spectrum in regards to four categories: Feel, Look, Do, Interest (Stern et al., 1987).Such an 'bipolar biological continuum' (Stern et al., 1987, p. 508) of gender allowed for more differentiated analysis but was finally revised, as from a queer-feminist standpoint at least the attribution of acts and interests as being on a bipolar continuum are questionable.Figure 1 shows a different approach to surveying gender and sex by Magliozzi et al. (2016).They chose to survey how people see themselves, the gender that most people ascribe to them, their assigned sex and gender at birth and their current gender identification (Magliozzi et al., 2016).Researches might refrain from using more complex scales because they fear it would be too time-consuming or too difficult; or they worry that being asked for the gender assigned at birth might be uncomfortable for some people or that especially open-ended questions regarding gender might lead to 'potential mischievous responses' (Fraser, 2018, p. 350).However, as those responders who choose to give untruthful answers often do so in more than one question (Robinson-Cimpian, 2014), mischievous responses to openended items on gender and sex might be useful to identify and exclude cases that shouldn't have been used anyway (Fraser, 2018, p. 350).
Laurel Westbrook and Aliya Saperstein argue, that if we continue to both essentialize and dichotomize sex and gender, survey research will continue to produce findings and reproduce beliefs that are disconnected not only from current social science theory but also from the diversity of gendered experiences 1 .(Westbrook & Saperstein, 2015, p. 536) At the same time, '[e]ven an innocently neutral question […] can prime gender' (Fine, 2011, p. 9): Reminding a respondent of their gender can lead to evoke gender-related assumptions and self-stereotyping (Sinclair, Hardin, & Lowery, 2006).Therefore, as with all survey items, gender-related questions, their relevance, their theoretical understanding and their position in the survey need to be discussed and justified instead of being 'the default' (Magliozzi et al., 2016).

Biases in sampling and testing
Two further aspects where biases might be introduced are the methods on sampling and testing (Else-Quest & Hyde, 2016): For example might a random selection sample, while being 'often held up as the gold standard of sampling' (Else-Quest & Hyde, 2016, p. 325), might often underrepresent intersectional groups.
People with disabilities, homeless, those who are living in shelters, jail, prison or who do not speak the dominant language are often excluded from large-scale samples.We do know however, that people in these groups differ regarding social attributes.For example, is a higher percentage of men in prison -especially BIPoC men (Pettit, 2012).
LGBTIQ youth, especially lesbian, bi and trans* youth, are more likely to live on the street or in shelters (Takács, 2006).
One of the main known biases we find in quantitative approaches is the so-called test bias.
Test bias refers to the differential validity of test scores for groups (e.g., age, education, culture, race, sex).Bias is a systematic error in the measurement process that differentially influences scores for identified groups.Bias can be internal (psychometric properties, test structure) or external (differential prediction/selection) to the test.(French, 2014, p. 6619) Silke Schreiber-Barsch et al. raise the question 'Whose voices matter' (Schreiber-Barsch, Curdt & Gundlach, 2020) regarding the inclusion of different voices, here especially the voices of people with learning difficulties, in large-scale assessments.This seems to be the essential question to ask and looking at the way data is survey and interpreted gives a clear indication whose voices appear to matter.

Carrying out calculations and making assumptions
After having surveyed the data, further decisions have to be made and many of them include a risk of further adding a bias.
In the following, I will illustrate the argument by using PIAAC data of six arbitrary European Countries (which included the income variables in the public use files and which share the Euro as currency): Poland, France, Germany, Greece, the Netherlands, and Spain.

Preparing Data
A common step in data preparation is to exclude certain groups from the data as they are either not relevant to the research question or they might introduce biases.When we, for example, try to determine a gender wage gap in relation to adult's competencies or educational attainments, we might exclude outliers regarding pay, i.e. those with such exceptionally high incomes that they tend to skew the results that are meant to indicate mean and average income (disparities).Regarding the monthly wages, studies might also exclude those without payed work or those who don't work full time as those appear to be reasonable ground for lesser (or no) payment.Examples and further reasoning for these can be found in Polackek ( 2004), Auspurg, Hinz & Sauer (2017), OECD (2017), or Heilmann, Gal & Grotlüschen (2020).
Table 1 demonstrates how different decisions influence gender disparities in terms of monthly income.While including all adults between the ages of 16 to 65 results in a pay gap of 72.9 percent, excluding those without payed work or with part-time payed work does leave out more women than men (OECD, 2017).At the same time, excluding outliers with more than 10 times the mean income (here: 2,960 EUR) does exclude few people with incomes that are not representative of the main population but has a major effect on the income averages and their gender disparity (c.f.Kwak & Kim, 2017).With regard to the scientific treatment of data, this point to the need for transparency in these decisions and data manipulations.Even if in some cases the exclusion of populations can be justified objectively (e.g. to exclude groups without income from income calculations), it seems necessary to examine the way in which gender-specific differences are subject to preliminary marginalisation.In addition, adult education research is often concerned with determining the (cor-)relations between educational qualifications or competence levels and their outcomes.The considerations so far often only prepare the ground for the further analytical steps -often linear regression models.

Assuming equal relations -Using mediators and moderators
Regression models are based on different fundamental assumptions.A often overlooked one is the homoscedasticity (Yang, Tu & Chen, 2019).
[186] Heilmann Homoscedasticity refers to the distribution of the residuals or error terms.If this assumption holds then the error terms have constant variance -in other words, the error for each observation does not depend on any variable within the model.Another way of saying this is that the standard deviation of the error terms are constant and do not depend on the explanatory variable values.(Tranmer, Murphy, Elliot & Pampaka, 2020, p. 36) Regarding gender, the concept of homoscedasticity can be used to describe how our models often overlook that the average relation between two variables might be distorted by gender.While for men, we can establish an average proportional relationship between their numeracy and literacy skills and their income and labour market position, this relation is not linear for women in the labour market (Heilmann et al., 2020).Additionally, the usual coding of gender as 0/1 (or 1/2) for 'male'/'female' (cf.ISO, 2004) and thus often handling 'female' as the deviation of the reference category leads to further invisibility of women in these analyses.To demonstrate this, table 2 compares different regression models.The models in table 2 and their coefficients, predict very different incomes.How a regression analysis works, in basic terms, is to average out the different effects of variables.Therefore, the coefficients of model 1 equal the mean distributions.The intercept indicates the men's average income and the coefficient shows that people that are invocated as fulltime working women earn an average of € 430 less.
Model 2 shows that men with a low educational attainment earn an average of € 2520 (see intercept for model 2).The average men with medium or a high educational attainment earn on average € 670 and € 2040 more per month.Averaged over all the educational groups, women earn about € 570 less.While this gives us a first indication of how educational attainments might relate or even impact monthly income, this model assumes that this impact is the same for men and women (c.f.Wu & Zumbo, 2008).
Only when we add the mediator or interaction term in model 3 we can see that the monetary benefit that might come with higher education differs for men and women.While men seem to benefit from higher education, women seem gain a lesser average from medium education attainments and a higher average with a higher education.Highly educated women earn an average of € 4580 2 while highly educated men achieve an average of € 4200 This demonstrates that the simple addition of a gender variable adds only little understanding of different life experiences of different groups of women.

Discussion
This paper looks at a few selected aspects of quantitative research in adult education and aims to demonstrate that common methods often marginalise women, keep non-binary people invisible, and disguise that in some cases men and women benefit disproportionally from factors like education.Neither gender group is as homogenous as simple averages suggest.A greater focus must be placed on alternative methods which offer a more diverse and intersectional view of different groups (for example on modelling of competence for gender and race: Hester, Payne, Brown-Iannuzzi & Gray, 2020; on intersectional effects of SES: Cascella, 2020).Instead of trying to include a multitude of variables into one or as little as possible regression models, a more contexualised and intersectional approaches might provide further insight (Scott & Siltanen, 2017, p. 378).By excluding vulnerable groups and assuming that different effects of education, race, etc. can be controlled for by averaging out their effects, we are compartmentalising gender disparities and therefore keeping them less visible as a whole.

Limitations
The arguments presented against the common regression models are neither new nor surprising.On the contrary, they are frequently cited, among other points, when fundamental arguments are made against quantitative approaches.There is no question that in the development of many large-scale educational data sets, gender questions are usually either completely disregarded or addressed in a way that is difficult to reconcile with current theoretical conceptualisations of gender.These surveys often do not include the relevant gender variables (Bowleg, 2008, p. 322).Nevertheless, if they do include the variables needed for specific research questions, commonly used methods often misrepresent societal structures.
Among the central counter-arguments against the proposed ideas is the supposed objectivity of figures, numbers, and statistics.This assumption, however, has been refuted in various places -instead, its deep embedding in societal power relations has been shown (e.g.Addey, 2018;Foucault, 2019).Similarly, one might argue that a complex understand of gender might over-complicate quantitative approaches and limit their practicability.If, however, this practicability is shown to systematically marginalise groups, we might call into question the legitimacy of such an argument.

Implications
This paper argues for more awareness and more critical approaches in quantitative research and in its interpretations.A broader feminist approach to quantitative research could improve how gender is commonly conceptualised and operationalises gender and diversity.By normalising the following 3+1 steps in our quantitative research, the potential of large-scale surveys could be better exploited and be used as a tool for our own critical research on gender.(1) By being more reflective of data sampling and collection methods and potential biases and by articulating these reflections as a necessary part of research (instead of feigning that our research is universal and objective) the results can be better embedded and interpreted.At the same time, this could potentially contribute to the establishment of alternative survey methods in the future.
(2) The choices that we make in manipulating our data and the selection of variables could be made more transparent and be discussed at a greater depth than currently usual.The presentation of the used method could benefit from a more detailed discussion of which biases lead to decisions and thus may be further reproduced or made invisible.
(3) The more habitual use and incorporation of mediators and interaction terms might improve the precision of statistical findings.They reveal greater complexity and are capable of incorporating intersectional relations even in relatively simple models and methods.As long as we cannot show or soundly argue that a variable does not intersect with our independent variables, we need to include interaction terms (or mediators) or be transparent about not doing so.
In order to see more diversity reflected in the major surveys in the future, it seems important to include modern questions of gender, diversity and intersectionality in quantitative research.Above all, however, it is relevant to (4) strive to broaden the perspectives of quantitative researchers, work on more ways to combine diverse theoretical concepts with quantitative methods and to try to make research teams more diverse.In order to raise awareness to issues of inequality and to strengthen feminist approaches and interpretations of quantitative findings we need diverse perspectives included in every step of the process.
Notes 2 This is the sum of the intercept and the coefficients for female, high ed., and female high ed.Thus the sum of the average income of a male with low educational attainment (€2700) minus the average difference between them and women with low educational attainment (who on average earn €1230 less) plus the average difference between men with low educational attainment and men with high educational attainment (which is €1500) and finally a corrector (interaction term) indicating how this difference between low and high educational attainment differs for women (€20). 3Similarly to above, this is the sum of the intercept (€2700) and the coefficient for high educational attainment (€1500).

Table 1 :
Differences in monthly income for men and women depending on the sample composition.Basis: First round PIAAC data from Poland, Spain, France, Germany, Greece, the Netherlands.The EUR-values are rounded to the nearest ten.

Table 2 :
3. Coefficients of four regression analyses on monthly pre-tax income of full-time working men and women; Basis: First round PIAAC data from Poland, Spain, France, Germany, Greece, the Netherlands.The values are rounded to the nearest ten.Ed. = Educational attainment.