Examining the Use of PhET Simulations on Students’ Attitudes and Learning in General Chemistry II

Chemistry is considered difficult to students to learn because many of its concepts are abstract in nature and require visualization at the sub-microscopic level of representation. Physics Education Technology (PhET) offers students the ability to understand and relate both chemical systems and what is happening at the sub-microscopic level through dynamic visualization. Simulations like PhET can be used as a powerful transformative tool for the teaching and learning of science. The research design and paradigm goal is to investigate the students’ perceptions on the impact of PhET simulations on their learning and attitudes and to identify PhET’s most helpful features. The data gathering tool in this research project is a survey that comprised of Likert-type and open-ended questions that was handed out to students who have completed General Chemistry II and were acquainted with PhET simulations as part of their laboratory sessions. The research took place at the City College of New York, an urban, minority serving, and public college. The number of research participants is 158. The implications of the research findings are PhET interactive simulations have an overall positive impact on students’ attitudes and perceptions about learning, PhET simulations promote students’ development of conceptual understanding of chemistry concepts and content, PhET simulations seem to promote and facilitate learning and understanding of abstract concepts, and PhET simulations furnish learning opportunities that otherwise cannot be attained in a traditional laboratory setting. The data presented in this paper support the notion that there is a need to update and modify general chemistry laboratories to reflect emerging technologies such as PhET interactive simulations.


INTRODUCTION
Laboratory experiences have continuously provided students with opportunities to interact directly with the material world through the use of various tools, models, equipment, and theories. While science laboratories are defined as areas where students follow a given procedure and use a variety of different equipment and techniques, student activities in a lab can vary. Laboratory education can be applied in a variety of different ways. One way is through physical manipulation such as dissections in biology, chemistry experiments, and other real-world materials that can be effected through hands on experimentation. Another is the use of simulations through computerized models that allow for interactions unlikely to be observed firsthand to be analyzed (Lunetta, 1998). Examples of such includes interactive biotechniques laboratories, observing molecules in chemistry, plant or animal systems. In a laboratory setting, students have the possibility to design experiments, handle equipment, observe and record data, analyze outcomes, and establish scientific reasoning (NSTA, 2004). Laboratories have long been deemed necessary in order to develop a deeper understanding of science and skills when it comes to the nature of science (NRC 2006, P. 127).
In a laboratory environment, students can also learn how to effectively work both independently and collaboratively in order to complete the task at hand, divide work, take on multiple roles, share ideas, and discuss results. Since laboratories should correlate closely to what is being taught in lectures, it can generate a better understanding of the concepts and theories through hands on experiences (Biehle, 1999;Bopegedera, 2011). Chemistry instructors underscore the importance the laboratory play in the teaching and learning of chemistry and as the place where students learn to appreciate chemistry as an experimental science. With the clear benefits students can acquire from work in a lab environment, there are numerous problems that students can face with when working in a traditional laboratory setting which include equipment failure, cost, availability of space, safety, motivation, and interest.
Chemistry educators have always been interested in improving students' laboratory experience. Some of these attempts include: making the laboratory sessions more interesting, relating the experiments to concepts covered in lecture, and integrating inquiry-based learning into the laboratory experiments. Traditional laboratory experiments fall short of promoting learning and conceptual understanding of chemistry concepts and do not adequately support learning. The "cookbook" procedure do not lend themselves to a learning investigation and there is emphasis on performing certain techniques without the development of understanding of science practices (Hofstein & Lunetta, 2004).
There is a growing need for chemistry students to learn and understand scientific practices as part of the laboratory component of courses by being immersed in an inquiry-based activity (NRC, 2012). PhET interactive simulations provide an opportunity for students to be involved in a guided inquiry-based learning activity through the analysis and interpretation of data. Research in science education favors inquiry-based laboratory experiments over traditional "cookbook" experiments (Lazonder & Harmsen, 2016). Research has demonstrated that inquiry-based learning positively impacts students' process skills and content knowledge (Bunterm et al., 2014).
Carl Weiman tried to tackle possible challenges educators may face in laboratories by creating a project called PhET Interactive Simulations. The purpose of this project is to provide a virtual laboratory experience through the use of animation and interactive environments that could drive students to explore reactions. This was founded in 2002, as a nonprofit organization, and is based in the University of Colorado (Adams et al., 2008a). While it initially began as an interactive site that focused on physics topics, it has branched out to offer a variety of options on topics in fields like chemistry, biology, mathematics, and earth science. With their mission statement to "advance science and math literacy and education worldwide through free interactive simulations," the simulations provided have been translated into 65 different languages. The idea of expanding the use of simulations came from the lectures he would give.
Computer simulations laboratories provide an alternative avenue to traditional laboratory which has been found to be more effective in promoting students' understanding of challenging concepts (Alsultanny et al., 2014). The simulations are described as computer-generated dynamic models that can promote learning of concepts through simplified models of real-world phenomena through animation, visualization, and interactive learning experience (Bell & Smetana, 2008).
Computer simulations such as PhET are widely available in science courses and are becoming an integral part of science teaching and learning and can be used to enhance traditional instruction and promote learning. PhET, which is an interactive simulation developed by the University of Colorado Borlder, can be effective in the teaching of chemistry and physics at the high school and college level (Perkins et al., 2012). PhET interactive simulation provides an alternative approach to the traditional laboratory and can enhance students learning through visualization, demonstrations and illustrations (Makransky et al., 2017).
The PhET Interactive simulations which include several chemistry simulations are offered freely to instructors and teachers though their website at the University of Colorado Boulder (https://phet.colorado.edu/). Each simulation is accompanied with several supplementary materials that immerse students in a guided inquiry based learning activity . It should be noted that PhET interactive simulations can be used as a tool for inquiry based learning (Smetana & Bell, 2011). Furthermore, PhET interactive simulations provide students with content support, process assistance, affective learning goals reinforcement .
In one research study, authors' data suggest that the interactive simulations are an effective implicit scaffolding technique through experimentation that does not overwhelm the students and provides an avenue for guided-inquiry learning (Moore et al., 2013). PhET simulations with opportunity to engage in an active learning exploration which might cause a change in their epistemology of the concepts (Bing & Redish, 2012).
PhET, or Physics Educational Technology, is a site that contains interactive simulations for science (physics, biology, chemistry, earth sciences) and math at elementary, middle school, high school, and university levels. Depending on which simulation, it can be run online from the website, or it would have to be downloaded. They could be useful as a lab or a homework assignment. Within this interactive site, there are visual displays and interaction between the student and the concepts being taught which helps to develop understanding (Price et al., 2018).
A virtual learning environment has been a more widely adapted form of learning as it enhances a student's experience both inside and outside the classroom. With the increase of technology use inside the classroom as a beneficial tool, teachers and educators have looked to incorporate it in as many ways as they can. By introducing simulations as a form of learning inside the classroom, there are a set of goals focused on the students that are supported by these simulations (Moore et al., 2013). These goals include the ability to engage in scientific exploration, which includes posing questions, designing experiments, and analyzing data. Also, developing conceptual understanding using models, cause-effect relationships, and representations. Other goals mentioned are to make connections to everyday life, view science as being enjoyable and accessible, and taking ownership of the learning experience (Moore et al., 2013).
This teaching approach utilizes technology to set up a web-based platform that aids in the learning process of multiple courses (Rutten et al., 2011). These virtual environments are interlinked with our modern day educational institutions as the use of technology has increased drastically. With this form of learning, a certain aspect of reality is simulated in a virtual environment that allows participants to explore what things would be like in the real world. High school educators have tried to adapt this new, fun way to engage students in their learning and reinforce topics learned in lecture for a deeper understanding (Couch, 2014).
Since many of the students are now tech savvy through various outlets of technology such as cellphones, computers, video games, they can be better engaged through simulations on the computer since it provides a different outlet than the traditional forms of learning such as a textbook (Couch, 2014). While the effects of simulations in education have widely been researched and evaluated, it is important to understand why teachers adopt these learning techniques in their classrooms and how are they applied while teaching (Price et al., 2018). Research data showed that more than half of the respondents cited the top four goals to be visualize science phenomena or science representations, develop conceptual understanding, engage in exploration and discovery or inquiry, and develop enjoyment or interest in science (Price et al., 2018). Three common features shared were visualizations, ability to manipulate or interact, and individualized experiences. Another common benefit discussed to using simulations is the ability to participate in activities that are not possible with the materials in a typical classroom environment.
PhET simulations have been successful in reaching large numbers of users in the K12 and college level with over 45 million runs per year and usage in all across the United States and usage in all 50 states. Although, the importance of these technologies are evident, it can be challenging to incorporate them to improve a student's performance. As the search for higher education remains a large concern in this country, many institutions have adopted the new virtual learning environments (Rutten et al., 2011).
In order to gather more information about the implementation of PhET simulation, a survey was conducted of more than 1,500 college and high school physics educators across America . Collectively, the acquired results indicate that PhET simulations are flexible tools used by educators to achieve various educational goals and respondents are using them with diverse populations that are diverse in ability, background, and major .
Computer simulations have been largely applied in science education to elevate the curriculum. The application of PhET simulations in a lab has many benefits to the overall experience. One of these benefits is new possibilities for different experiments. Since not everything is possible in a classroom with the equipment given, simulations allow for students to experiment with and engage in activities that would otherwise not be possible or practical in a real setting (Wieman et al, 2010). By providing more and various kinds of simulations, students can tackle more concepts that possibly could not have been thoroughly observed in a laboratory. Another benefit that comes with the use of simulations is quick repeatability. Students have the capability of repeating the experiment or activities multiple times in order to better understand the experiment or to try and test out the experiment under different conditions. A simulation would be able to show things more clearly and in a real world scenario which can help students grasp concepts and understand how things should be before being exposed to a messier world . With simulations, scientific models are designed to be visible in order for students to grasp not only what is happening, but why is it happening. Also, students often enjoy the use of PhET simulations and find it more engaging and beneficial.
Science laboratories and the design of the experiments at the university level seem to raise some questions about achieving student expectations and reaching faculty goals, an alternative learning environment could create a better learning environment (Bortnik et al, 2017). Virtual labs tend to be learner-centered and inquiry-based which allows an enhanced way of thinking about the concepts and better retention for what was taught which is a really important skill to have when learning about the in depth theories and concepts of chemistry. This suggests that e-lab is an effective form of learning for students and has allowed students to equally develop skills and understanding that is shown in hands on experiment.
Computer simulations, like PhET, are commonly used with the expectation that it will allow them to devote more time to the students learning the material as opposed to setting-up and supervision of experimental equipment. It also allows for testing a hypothesis that has been made, manipulating various variables, and having multiple representations to understand the information (Adams et al., 2008b). In science labs, simulations largely contribute to improvement in a student's interest in the subject matters covered in their lectures. It also helps students build a conceptual understanding of the science, or what is difficult to see, with the guidance of simulations. Students are then able to build a mental framework about the concepts and construct their own understanding of what is being taught (Adams et al., 2008b).
Some of the reported advantages of computer simulations use in laboratories include: ease of use, ability to save instructors time to be devoted to students learning instead of equipment set-up and student supervision, availability of large set of variables to test and manipulate without worrying about safety, supportive learning environment though use of multiple representations (Blake and Scanlon, 2007). One recent article reports that PhET interactive simulations in physics develops students high order thinking skills (Yusuf and Widyaningsih, 2019). Additionally, PhET simulations provide students with scaffolding approach which can improve their learning (Erlin Eveline et al., 2019).
One of the challenges that students face to learning chemistry is the development of understanding and the relationship between the three levels of representations: macroscopic, symbolic, and submicroscopic. The ability to navigate between the three levels of representations plays a significant role in learning and understanding (Smith & Villarreal, 2015). PhET interactive simulations allow students to engage in the three levels of representations: symbolic, macroscopic, and sub-microscopic and to relate and integrate the three levels in the learning process of chemistry .

METHODS AND EXPERIMENTAL PROCEDURE
The purpose of this research project is to investigate the impact of PhET on students' attitudes and learning. The research project took place at the City College of New York during the spring and fall semesters of 2020. The City College of New York is an urban, commuter, public college, and a minority serving institute. The participants were enrolled or have completed General Chemistry II. We used a survey made up of Likert-type and open-ended questions as our method of data collection. The survey was given to two experts who agree that the questions adequately capture the investigation about solubility. The reliability coefficient was assessed to be 0.81 through the use of test-retest reliability method. Additionally, interviews were carried out on a subset of the participants to illicit more information and clarifications, which was valuable in creating the rubric and coding the data. The survey was administered and collected from 158 participants in accordance to the Internal Review Board (IRB).
Although the development of PhET simulations began with physics, animations and simulations have long been viewed as significant in both the teaching and learning of chemistry . Due to the abstract concepts that are taught in chemistry, students can often experience multiple problems with understanding the lessons taught. As a result, it is crucial for students to construct the ideas on their own and laboratories allow for them to apply their scientific knowledge. Laboratory assignments are crucial in order to help students build their own understanding by being hands on. For General Chemistry II, there are many theoretical concepts that can be understood and embraced further with the use of PhET simulations in the lab. Since these extensive topics cannot be thoroughly discussed in lecture, having students use interactive simulations can improve performance on those subjects. Since these can also be completed outside of the lab, students are able to work on them and practice to increase understanding on the subject matter.
The PhET interactive simulations relay on students' learning through exploration and experimentation. The PhET interactive simulations provide students with the opportunity to relate the three levels of representations which are the symbolic, macroscopic, and sub-microscopic within a single experiment. Additionally, the simulations are interactive, exploratory in nature, engaging, and provide student with opportunities to find causality and effect relationships.
We wanted to assess students' attitudes towards PhET chemistry simulation learning interventions and the impact on their learning.

Guiding Research Questions
Our research was structured to address the following specific questions: 1. What are the students' perceptions on the impact of PhET simulations on their learning and attitudes? 2. What are the students' perceptions on the role of PhET Simulations in developing conceptual understanding? 3. What are students' perceptions on the benefits of PhET simulations for their learning?
The Likert-type questions were on a five-point scale ranging from strongly disagree to strongly agree and were converted to numerical values as follows: Strongly disagree (1), disagree (2), neutral (3), agree (4), and strongly agree (5). A single factor ANOVA method was performed on the Likert-type questions part of the survey and found that P<0.05 which indicates evidence against the null hypothesis and that reveals a strong relationship between variables. Furthermore, the data analysis shows that the mean-square between groups is 2.933 which is significantly larger than the mean-square within groups of 0.936. The ratio between groups-mean square and within-groups mean square is 3.681 which is large enough to reject the null hypothesis with confidence. The data was entered into a spreadsheet and the average value from the respondents were calculated.
For three of the open-ended questions, a rubric was used to convert the students' responses into numerical values ranging from 1 to 5, where 1 stands for a negative answer and 5 for a positive response. An example of the rubric used for the open ended questions about the overall experience is given points as follows: A response that was completely negative and accompanied by an explanation of why the student's experience was negative received a score of (1), a response that was negative in its entirety but did not provide an explanation as to why the experience was negative received a score of (2), a response that was comprised of both positive and negative comments was considered to be neutral (3), no weight was given to any aspect of the response, a response that was positive but did not offer an explanation as to why the student's experience was positive yielded a score of (4), and a response that was completely positive and accompanied by an explanation of why the student's experience was positive received a score of (5). The data was applied by the two researchers and there were 95% agreement. The small fraction of differing numerical value assignment was no more than 1 and a discussion was used to come to consensus. For one of the questions, a pie chart was created based on the responses and their percentages.

RESULTS AND DISCUSSION
The results presented in Table 1 suggest the students perceptions of PhET simulations is that they overall agree that PhET labs helped them better understand concepts covered in lecture, made covered content makes more sense than it did before the simulation, promoted their understanding of the topics covered after the PhET simulations, and facilitated their understanding of chemistry and how it works. In one study on a physics course using simulations, researchers report that the simulations caused an improvement in students' interest in the course and their academic performance (Baltzis and Koukia, 2009). In chemistry, integrating computer simulations into laboratory sessions caused an improvement in content knowledge (Limniou et al., 2007). In one research study, researchers found that PhET simulations improved levels of confidence about content and increased their understanding (Watson et al., 2020).
Students also seem to agree that PhET simulations were clear and easy to follow, and provided a fun learning experience, gave new learning opportunities that were absent in traditional laboratory settings, improved conceptual understanding, contributed to grade improvement, and was an overall positive learning experience. This is consistent with research in science education that use of PhET in the teaching and learning of STEM courses can improve the students' positive response to the learning (Perkins el al., 2014). Research in science education also reports that PhET interactive simulations have been found to improve explanations of abstract concepts in physics and promote learners' thinking abilities (Wieman et al., 2010). Furthermore, research has shown that PhET interactive simulations have been reported to students' learning performance (Debowska et al., 2013) and PhET simulations have been reported to improve students' conceptual understanding (Clark and Chamberlain, 2014). Additionally, researchers report that PhET simulations provided students with experience that is joyful and autonomous, promoted a positive attitude towards chemistry, and nurtured their ability to visualize concepts in chemistry which results in improving conceptual understanding (Penn & Ramnarain, 2019). Figure 1 is a bar chart depiction of the average answer from respondents for the open-ended questions part of the survey. The data suggest that students had a positive experience using the PhET simulations and that the PhET simulations helped them develop better understanding of the concepts covered in lecture. This is consistent with research in science education that states virtual laboratory simulations can be used to promote students' conceptual understanding and improve students' attitudes about the content where students can experiment, visualize, and construct representations that can help in improving long term memory of what is being learnt (Hsu et al., 2017). In one study, the authors reported that the use of PhET led to a positive learning experience for the students, as well as, worth the time and effort by the students (Correia et al., 2019). Furthermore, in one research study, authors report that PhET simulations improved conceptual understanding and motivation in High School physics class (Prima et al., 2018). Students also agree that they would recommend integrating PhET simulations into other science courses. Online simulations have been found to improve students' engagement in learning of science complex concepts more so than traditional chemistry laboratories (Wu et al., 2013). Figure 2 is a pie chart of students' perceptions about the features that are beneficial in PhET simulations. The data show that 36.6% of students' report that they found the ability to manipulate and play around with variables as most beneficial to their learning. This finding is consistent with other research that reports simulations can provide a platform for student to emulate what scientists do through including observation, data collection, experimentation, data analysis, and scientific reasoning (Kim and Hannafin, 2011). Additionally, 23.2% of students agree suggest that the PhET simulations allow for interactivity aspect of the learning process which is essential to their learning. This is supported by other research that reports simulations promote active learning by providing students with the ability to study patterns and complex systems by manipulating certain variables (Lindgren and Schwartz, 2009). Furthermore, 26.8% of participants suggest that diagrams and graphics were beneficial due to their impact on the students' visualization of the concepts presented. Finally, 13.4% of students report that ease of use and clarity of instruction were the most beneficial. Now that I've virtually applied PhET simulations, covered content makes more sense to me than it did before the simulation.

3.91
I have a better understanding of the topics covered now that I have PhET experimented with it.

3.88
Virtually performing PeET labs helped me better understand how chemistry works. 3.83 The PhET simulations were clear and easy to follow. 4.06 The PhET labs were an overall fun experiences. 4.03 Virtual Labs (PhET) provided new learning opportunities I would otherwise not have experience in traditional lab 4.08 The PhET labs helped me to better understand concepts covered in lecture.

3.80
The PhET labs were overall beneficial to me and contributed to my improvement in the course grade.

3.71
Please rate your overall experience with this lab from 1 to 5. 4.05

CONCLUSIONS
The data presented in this paper suggest that PhET interactive simulations had an overall positive impact on students' attitudes and perceptions about learning. PhET simulations provided the tools that helped students develop better understanding of chemistry concepts and content covered in lecture. PhET simulations seem to promote and facilitate learning and understanding of abstract concepts. Additionally, PhET simulations provided clear instructions and were easy to follow, and furnished learning opportunities that otherwise cannot be attained in a traditional laboratory setting.
There are several aspects of PhET simulations that our students seem to value. These include: the ability to manipulate and play around with variables, the interactivity aspect of the learning process which is essential to learning, the diagrams and graphics due to their impact on the students' visualization of the concepts presented, and the ease of use and clarity of instruction.
The data presented in this paper support the notion that there is a need to update and modify general chemistry laboratories to reflect emerging technologies and reach students in their comfort zones. The PhET laboratory simulations allowed for a seamless transition to online learning as a result of COVD-19. We plan to continue to modify and update our laboratory offering based on faculty members' and students' feedback to reflect emerging