Keywords

1 Introduction

The recent Italian Educational Reform introduces computational thinking into the schools, starting at the primary level [20]. The schools are preparing for the introduction of this new subject: (1) in-service teacher training on computational thinking and educational robotics activities; (2) the purchase of hardware and software needed to employ educational robotics in the classroom. At present, the schools are oriented towards solutions proposed by manufacturers of educational robotics [16] that offer increasingly interesting and attractive modular robotics kit to the world of education. However, there are other solutions less familiar to primary school teachers, such as handmade robotics with recycled materials. Based on constructionism theory, handmade robotics, which is, is considered one of the most fruitful approaches for learning and teaching technology [25, 27]. This evaluation concerns mainly high school students as there has been little use of Arduino in primary schools [19]. For this reason, we have designed an educational activity for primary schools, called Rospino [28], based on the Arduino platform [1]. This consists of a recycled materials kit accompanied by software that we have developed which is freely downloadable from the Web [28]. This proposed activity presents a challenge to primary schools because it offers a different approach than kits composed of manufactured building blocks, which have been described as “predefined plastic shapes, user manufactured plastic shapes, laser cut shapes and a combination of craft and LEGO” [29, 34]. These “tend to restrict design freedom as they provide a set of predefined physical shapes that could only be assembled in specific ways. Crafting using everyday objects as primitives shapes provides more freedom in creative exploration” [34]. A common feature of handmade robotics projects is the use of recycled materials chosen to see if they incentivise equally the involvement and participation of boys and girls in primary school (from 9 to 12 years old). This question is relevant if we consider that currently at the level of international research there is still little investigation of gender differences in relation to computational thinking and coding [12, 21, 33] and little is known of handmade robotics [9, 19]. Although there are no precise figures on the use of Arduino in schools, one review [14] of the Arduino platform, which was used in our proposal, revealed that females constitute less than 1% of users in high schools; though, the e-textile fab lab communities has recently attracted a significant number of female adolescents and women [13]. In addition, although there has been research on the cognitive aspects of the writing of coding and on the use of these platforms in order to acquire new skills, there is limited evidence about gender differences in attitudes, beliefs and perceptions, as compared to the level of involvement in educational activities as a whole [13, 19]. For this reason, the exploratory study presented in this paper aims to investigate differences in self-efficacy and learning engagement among students being taught handmade robotics. In particular, the research questions are: (1) are there gender differences in students’ beliefs regarding self-efficacy after a Rospino robotics activity compared to before? (2) are there gender differences in learning involvement at the cognitive and perceptual levels during Rospino robotics activities?

The exploratory study was carried out on a total of 153 children in five primary schools in the northeast of Italy. It was presented as a relevant initiative, in education and pedagogy, to expand research studies in this sector. It was also intended to encourage interested schools, teachers and students to participate in handmade robotics educational activities. We present in Sect. 2 the characteristics of Rospino and the research conducted so far with teachers and children. Sections 3, 4, 5, 6, and 7 are devoted to presenting this research.

2 Rospino: Design Features

The Rospino project started with the aim of providing students an educational and economical kit of recycled materials (cardboard tablet, bottle caps, rubber bands, sticks and wooden cubes) to construct a basic robot, in order to stimulate their imagination and creativity. Throughout different development stages, Rospino has been enriched with other components that allow teachers and children to customize the robot movements: Arduino, breadboard, wires, batteries, servo motors and USB cable [28]. Together with these materials, software has been developed to simplify the robot programming [28]. Since this software has to be used by primary school children, it has been simplified as much as possible, following the principle established in the world of architecture which says “less is more”.

2.1 Research and Development

The First Prototypes. All design decisions for Rospino are based on research and feedback from children and teachers, together with developmental psychologists and pedagogists. Over the past two years, the design of Rospino has had several iterations with the end-user. It has been tested in many public schools and in the laboratory.

Research with Teachers. From near the end of 2015 through early 2016, data was collected from 14 primary and middle school teachers; additional data was collected from 150 more primary and middle school teachers through 2016 and early 2017. Both groups of teachers were questioned about their perceptions, attitudes, beliefs and experiences of robotics in order to modify and improve the design of Rospino. These teachers were exposed to a Rospino prototype and took part in training sessions, semi-structured interviews, group discussions and questionnaires to help researchers improve Rospino software and kits. Teachers were generally enthusiastic about the use of handmade robotics for professional development and for educational activities in their classrooms, although they were novices in this subject (as many of them came from the humanities). However, they also made several suggestions to improve the original design of Rospino, including: (1) they asked to simplify the procedure to connect the robot-artefact with Rospino software; (2) since the construction of mechanical/electronic parts requires several steps and novice teachers of robotics find these difficult to remember, the employment of a video was indicated [4]; (3) finally, some teachers suggested that the use of Rospino kit might be an activity more “attractive” to boys than girls, while no such comment was made about the software.

Research with Children. Through 2015 and 2016, we held a number of sessions with non-selective groups of ten children each, aged from 9 to 10, from whom we collected the following feedback: (1) the parts required for the robot’s construction should be easily manipulated and intuitively easy to connect; (2) the programming of the robot should be minimal as should the quantity of computer equipment needed; and (3) the recycled “decorative” material which is used to cover the robot skeleton should be designed and coloured more attractively. In these sessions, the children did not perceive or label the activities as boy things or girl things.

3 Present Research

From the experience gained so far, we concluded that although the Rospino kit might not be gender-neutral, no such opinion was offered about the Rospino software either by the children or their teachers. Therefore, we decided to try to determine whether the proposed handmade robotics is viable. To this end we took into account various studies which have been conducted to determine gender compatibility [5, 19, 34] in the educational field. We designed an educational workshop based on verbal, visual and computational storytelling activities [4] – employing group work and problem-based learning – in order to see whether gender differences emerged. To understand the impact of the educational robotics proposal, we conducted studies that were designed to answer the following research questions: (1) are there gender differences in students’ beliefs regarding their self-efficacy after a Rospino robotics educational activity compared to before? (2) are there gender differences in learning involvement at the cognitive and perceptual levels during Rospino robotics activities? We conducted the study on a total of 153 children, from eight classes in five primary schools (see Sect. 5).

3.1 Gender Differences: Beliefs, Expectations and Perceptions

Our efforts to evaluate the effectiveness of the proposed handmade robotics (Rospino) and to determine gender differences in this regard follow from the goal of promoting educational interest in computational thinking. We were inspired in these endeavours by research milestones, such as Weiner’s attribution theory [32], Bandura’s self-efficacy theory [3]; Covington’s self-worth perspective [10]; Atkinson’s expectancy/value theory [2] and Eccles’ achievement-related choices [11]. An acquaintance with these concepts helps one to see how students’ choices are influenced. Among the many possible lines of inquiry, we decided to investigate expectancies for success and perceived engagement in a particular task. Expectancies for success are based on how much trust an individual attributes to his ability to succeed in a particular task, while perceived engagement is determined by the degree to which an individual perceives that task to be important, useful, and engaging in itself. Accordingly, we have carried out an initial survey aimed at investigating self-efficacy beliefs and the perceived engagement of children. As the relationship between gender, self-efficacy and perceived engagement in the field of educational robotics has not yet been sufficiently explored in primary schools, this work is intended as a contribution to filling that gap.

3.2 Self-efficacy and Gender

Expectations of success have their genesis in the concept of self-efficacy. Bandura [3] defines self-efficacy as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances”. Research in this field has shown that students with high self-efficacy are able to persevere longer in their tasks, to demand more significant results from their research, to withstand the anxiety of schoolwork and achieve better results. It is important to note that there are no well-documented gender differences in STEM on self-efficacy.

3.3 Perceived Engagement and Gender

Self-efficacy is one of several factors that can affect the self-perception of students regarding their expectations of success. Childhood development from 9 to 12 is characterized by crucial physical, emotional, cognitive and social changes. At this stage, “girls [are] generally showing a decline in self-efficacy beliefs” [24]. To better under-stand this situation, we considered it essential to investigate aspects of perceptual and cognitive involvement in the learning of handmade robotics (User Engagement Scale [22]), together with the concept of self-efficacy [3]. In fact, it is necessary to explore the perceived engagement differences in cognitive capacity and problem-solving ability in girls and boys, and to combine this with affective variables, such as self-efficacy, which often are decisive for the selection of future studies and also are a primary source of gender differences with the passing of years. At this point, the relationship between self-efficacy and perceived engagement in relation to gender in the field of computational thinking has not yet been sufficiently explored.

3.4 Video Behavioural Observation

To further verify the students’ experience of coding with Rospino, we decided to analyze videos of the operations the students performed during the workshop. The purpose was to examine what students learned in practice and what their difficulties were. The criteria used to evaluate the videos were based on the educational goals of the workshop: (1) knowing how to use the sequence structure; (2) been being able to distinguish a command by its attributes; (3) during the testing phase: been being able to distinguish between hardware problems and software problems.

4 Robotics in Primary Education

The international research literature on computing education continues to confirm a concerning under-representation of women in the IT sector [14, 30] and worrying results about gender impact [23]. In the recent past, computer use and activities related to ICT were seen as a field of “male domination” [7] and this perception continues today, despite slight improvements in gender balance [26]. Even though recent research [23] confirms the persistence of the gender gap, there is strong evidence to support the thesis that teaching educational robotics to students can have a positive effect on their subject choices [6]. Research indicates that the use of cross-disciplinary pedagogical classes which blend art, narration and technology advances the balance of gender participation [8, 29]. Rather than limiting themselves to a particular task-oriented application, girls displayed an interest in technology primarily as a teaching method for stimulating the development of creative and expressive skills. Having participated in this form of education, girls have a better chance of becoming students in STEM [15]. So far, the most interesting results for girls appear to be obtained with computational e-textile toolkits Arduino LilyPad [9] which employ soft materials and incorporate crafting techniques, such as sewing cloth, rather than employing motors and gears [5, 13]. However, other research which measured student engagement among ubiquitous computing platforms (such as, Scratch 2.0, Lego Mindstorms ΝΧΤ, and Wearable Arduino LilyPad) found “that girls were emotionally engaged in robots as much as boys” [19]. In addition, the girls showed more interest in the robots than in Lilypad [19]. Avoiding historically feminine expressions and materials has led to encouraging results being achieved by other solutions, such as HandiMate (based on Arduino Nano) [34]. As regards self-efficacy, several studies have also found that girls’ confidence levels increase with practice [19]. Nevertheless, despite increasing research in the field, there is little evidence to show that the students’ perceptions, beliefs and attitudes are predictive of future choices [5]. Therefore, it is important to evaluate the consequences of programming computer systems with regard to students’ attitudes and intentions to study programming in the future.

5 Methods

We conducted a small scale study to investigate possible differences in the impact of handmade robotics activities. Below are details of the participants, conditions and measures.

5.1 Participants

A total of 153 primary school students, comprising 87 males and 66 females, enrolled in this pilot study. The study was held during school hours. The students were aged from 9 to 11. The majority of the students had previous experience of software usage, but not of software development. Only three pupils had really tried to program, by participating in school robotics projects. Before proceeding with data analysis, incomplete questionnaires were eliminated. In addition, since the coding activity took place on different days, it sometimes happened that a student participated in only a part of the workshop, due to absence. In these cases, questionnaire answers were discarded. For this reason, only 133 students out of 153 were analyzed; this group comprised 75 males and 58 females, aged from 9 to 11.

5.2 Conditions

Educational activity was introduced for each class through the narration of a story, followed by a phase of drawing and construction of a robot, in which students plotted a few characters from the story and built one robot with Rospino kit. Rospino software was later presented to the children, after which they were left free to explore the development environment and to program certain movements of the robot. Finally, the students were invited to move the robot in a predetermined path, which they repeated in subsequent tests.

5.3 Measures

The questionnaires administered were: (a) pre- and post-test on the sense of self-efficacy; (b) post-test on cognitive and perceptual student involvement. These questionnaires were preceded by the compilation of data on gender and age, on the possession of a home computer, and on computing experience.

The self-efficacy, pre- and post-activity questionnaires administered to students employed the four-item Likert scale of five steps (ranging along an axis from (5) “completely agree” to (1) “completely disagree”). It investigated four aspects: (i) level of self-efficacy in learning activities and understanding of coding; (ii) expectation of doing well in activities one chooses to pursue; (iii) security of concluding one’s chosen activities successfully; (iv) security of achieving excellent results from the activity of coding. The second questionnaire, which was administered after the workshop on robotics, was based on the User Engagement Scale. It employed the eight-item Likert scale of five steps, ranging from (1) “completely agree” to (5) “completely disagree”. This questionnaire was intended to measure the level of students’ engagement in their coding activity. The User Engagement Scale consists of a wide variety of questions, but for our sample formation and the context in which the activity took place, it was decided to limit the items to eight in order to investigate the following aspects: focused attention; novelty; involvement; aesthetics; perceived usability. These measurements were aimed at assessing the differences in effort perceived by boys and girls during the workshop.

6 Research Results

The results reported below are based on a non-probabilistic sample, as we were obliged to sample only that part of the population which was accessible in the circumstances. These can be characterized as ad hoc samples, drawn from those schools for which we were authorized to enter the classrooms and restricted by the availability of teachers to subject their students to research. For this reason, the validity of our sample is closely linked to the situation to which it refers and there is no guarantee of its validity in different circumstances.

6.1 Results Regarding Changes in Self-efficacy

Before analyzing the results, the reliability of the questionnaire was estimated by calculating Cronbach Alfa and obtaining an acceptable result (α = 0.70); this data was confirmed by the inter-item correlation matrix (all values positive) and the corrected item-total correlation, which demonstrated good internal consistency among the questions. To verify whether the educational activities had produced an improvement in the sense of self-efficacy, a t-test for dependent samples was performed on data from 133 students; this revealed that the change (Pre: M = 4.14, SD = 0.56, range from 2.50 to 5.00; Post: M = 4.25, SD = 0.57, range from 2.25 to 5.00) was statistically significant (t(132) = −2.06, p < 0.05). Analyzing data split up by institution (see Table 1), it can be seen how the increase in self-efficacy was statistically significant only in two schools: School 2 (Pre: M = 4.24, SD = 0.59; Post: M = 4.51, SD = 0.32; t(23) = −2.27, p < 0.05) and School 4 (Pre: M = 3.86, SD = 0.58; Post: M = 4.24, SD = 0.66; t(17) = −2.37, p < 0.05).

Table 1. Self-efficacy per school
Full size table

Deeper analysis of these results, concerning School 2 and School 4 (see Table 2), put into evidence how the improvement of self-efficacy was statistically significant in female students, as highlighted performing both a t-test for dependent samples and a Wilcoxon test. More than this, it is interesting to underline that the level of self-efficacy in females was lower than males in the pre-test, while it is the same or higher in the post-test, and the same thing happens in other schools. This is very interesting if we consider that at this age (9–11) girls are more self-critical and less secure than boys.

Table 2. Self-efficacy per school and sex
Full size table

6.2 Results Regarding Perceived Engagement

After the workshop on coding a questionnaire consisting of nine items was administered to the children. This questionnaire was based on the User Engagement Scale [22] and was intended to measure the degree of student involvement in the activity of coding. The research collected data on the following aspects: focused attention, novelty, involvement, aesthetics, perceived usability.

Hypotheses. This study aims to determine whether there is a gender difference in the cognitive involvement of students in coding activities. If there are differences between the male and female populations, we should be able to reject the following assumption:

H0: the median of the population from which the sample of males is extracted is equal to the median of the population from which the sample of females is extracted: therefore, there is not a gender difference in cognitive student involvement.

Results. The class in which the experiment took place was drawn from primary schools; it was composed of 133 students aged between 9 and 11 years. From this group, two independent samples were created: one of 75 males and the other of 58 females.

Before analyzing the results, the reliability of the questionnaire was estimated by calculating Cronbach’s alpha coefficient. The calculation of this index demonstrated a fair internal consistency of questions (α = 0.74).

Since male data and female data are not distributed in a Gaussian way, instead of calculating a t-test, the Mann-Whitney statistical test was performed to analyze the results, reporting the following results: Mann-Whitney U = 2043 and relevant probability p = 0.548, which corresponds to Asymp. Sig. (2-tailed). Through these values it was possible to calculate the effect size requivalent, amounting to a value of 0.053, interpreted as a negligible effect size.

Data Analysis. Taking into consideration the results reported in the previous section, it is not unlikely that the observed data is the result of a true null hypothesis (H0 given in section “Hypotheses”): this is why we accept it.

The results suggest that most likely the male population (n = 75, median = 4.00, Q1 = 3.43, Q3 = 4.71) has a median score in the questionnaire concerning user engagement equal to that of the female population (n = 58, median = 4.14, Q1 = 3.43, Q3 = 4.57); therefore there is no empirical evidence of a gender difference (U = 2043, z = −0.601, p > 0.05, r = 0.053).

6.3 Analysis of Results of Video and Programs

The analysis of the data was intended to evaluate the achievement of the objectives described in Sect. 3: to detect the difficulties faced by the students during coding with Rospino software, and to collect information regarding students’ involvement and their reactions when tackling obstacles. The results of the analysis follow: (1) more consecutive blocks “go ahead” (lack of optimization); (2) overload operation “turn right”/“turn left”, which is also associated with the operation “go ahead”; (3) lack of a strategy for the resolution of the problem (many trials and lack of planning). Moreover, videos put into evidence the students’ main difficulty in robot assembly, that is to understand the differences among the three lines: input, power supply and ground. The issues listed above affected both males and females in the same way.

7 Discussion and Limitations of the Research

Based on the statistical results it could be stated that there are no differences between males and females. However, in our opinion, the interpretation of results should not be limited to the level of statistical results. We must take into account the psychological development that characterizes this age range and is different for males and females. In fact, adolescence for girls begins between 9 and 11 years, while for boys between 11 and 13 years [18]. Adolescence, more than the other stages of the life cycle, has been defined as a transition period characterized by profound and significant cognitive, affective, social, and physical changes which by challenging beliefs, attitudes, and thoughts affects the identity of teenagers and inevitably their security [31]. In adolescence, the maturation of analytical and introspective capabilities and the definition of one’s own identity allow for a progressive reorganization. This process does not lack tensions and often includes complex issues that are still under exploration, especially in this increasingly digital society. Specifically at this stage of life, several studies of girls and boys of the same age, show that girls suffer a fall in self-efficacy [24] in the field of science compared to boys, and this gap increases as they grow older [17]. This data can also be seen in the self-efficacy pre-test results of our group of girls (see Table 2). However, we can notice that in the post-test the self-efficacy of girls has improved (in some cases it is equal to or greater than the boys). Therefore, in interpreting data, we must take into account the differences in development between males and females (mentioned above) and assume that girls have filtered and processed questions more critically and with a lower sense of security than their male colleagues (the pre-test confirms this hypothesis). Therefore in answering the research questions we can state that: though being more “self-critical” or “less secure” than boys, we can infer that girls show meaningful improvements in learning engagement and self-efficacy tests, equal to or better than boys. Therefore, our interpretation should be based not only on the statistical results but also on the psychological differences of males and females of adolescent age.

We can conclude, in this first phase, that the introduction of robotic craft activities has the potential to attract both girls and boys as active participants, despite the fact that they have used decisively non-material forms and expressions that historically have a more feminine orientation. These results were positive [5, 13], as the original feedback received from teachers, which had perceived robotics as primarily an activity for males, was contradicted by the children’s self-reporting. In fact, we observed that after the initial indications given in the classroom, both males and females developed a sense of autonomy in the design and creation of their robots. The children assembled different materials with great dedication, making several hypothesis about what were the best building solutions. They paid careful attention to their artefacts’ physical and aesthetic details, which resulted from their own creativity and from experimenting with their robots on the floor. They showed perseverance in physically improving their “creatures” (e.g., by looking for materials other than those proposed) and in using the software to program the paths of their robots on the floor (i.e., choosing the initial path of the robot and changing it according to the physical conditions it encountered). The children showed tenacity in achieving, through testing and retesting, their goal of making their robots do what they had predetermined. They felt that the activity was “very cool!”, “an adult thing”, “work for engineers”.

The work has some limitations: we cannot exclude that previous experiences or implicit and explicit beliefs or gender stereotypes (transmitted by family, friends, or teachers) could have weighed on the thinking of the girls and boys. Although the workshop took place in curricular hours, the experiment was required to coexist with other school activities that required the children’s attention and commitment. The novelty factor might also have influenced the degree of involvement of the children, as it emerged from the questionnaire.

For future work, we intend to strengthen this explorative study with longer periods of intervention and to enhance the pedagogical approach by employing cross-disciplinary studies. We will compare our proposal for handmade robotics with other educational robotics solutions, and we will deepen further exploration by qualitative research methods, in order to explore the children’s expectations, attitudes, beliefs, and perceptions. To better understand the beliefs of children and the possible obstacles they encounter in their school career choices, we would like to explore the beliefs and stereotypes among other educational actors, such as teachers and parents, about school-based robotics activities.

Note: for reasons of national assessment of Italian university research, the authors must declare which sections each has written, in spite of the fact that work is entirely the result of continuous and intensive collaboration. Sections 1, 2, 3, 4 and 7 are by M. Banzato. Sections 5 and 6 are by P. Tosato. The authors would like to thank Matthew Hoffman for valuable comments.