Bibliometric Analysis: STEM Research in High Schools (1988-2023)

STEM, an educational model, equips gifted students with essential skills to foster creative problem-solving. The aim of this study was to capture the STEM-related landscape in high schools. The method used is bibliometric analysis. The data retrieved comes from the database Scopus. The results show that STEM-related publications in high schools continue to increase every year, especially from 2016 to 2021. Publications in 2011 have cited more than 160 citations, this number is more than any other year. Indonesia is the most influential country in this field. The focus of research related to this field is 1) outcomes and motivation; 2) critical thinking and 21st century; 3) problems and development. The new theme for this field is critical thinking, 21st century, project-based learning (PjBL), covid and perception. Keywords that become new themes such as critical thinking not yet connected directly with keywords with other new themes such as perception. Keywords that were the primary focus, such as motivation, outcomes, and STEM in high schools, did not directly align with the emerging theme of Covid. This novelty offers valuable insights for future research.


INTRODUCTION
In the world of education, STEM is needed to meet the needs of the 21st century, namely to support learning that is supported by technology, and students have knowledge (Harun, 2020). STEM is an integration of science (S), technology (T), engineering (E) and mathematics (M) that are related to work experience and everyday life (Hafni et al., 2020). STEM is integrated learning which consists of four aspects, namely science, technology, engineering, and mathematics (Dier & Asrizal, 2022;Gao et al., 2020;Isdianti, Nasrudin, & Erman, 2021;Laila & Asrizal, 2021). So, STEM is a learning approach that combines science, technology, engineering and mathematics.
According to Winberg et al. (2019), STEM is a term that is more commonly used at the school level than in higher education, STEM is an educational model suitable for equipping gifted students with the necessary skills, developing creative and original solutions to problems, and directing them to STEM-related professions (Kocaman, 2023). STEM is a learning approach that is considered capable of building students' higher-order thinking skills (Hakim et al., 2020). STEM was introduced as an area in the High School program in several countries (Pawilen & Yuzon., 2019). Effective STEM education relies on active learning activities with a constructivist background (Mystakidis & Christopoulos, 2022).
Stronger STEM Education development research has become a trend in recent years (Calvo-Villamañán, San Millán, & Carrilero, 2022). Currently, various reports have been published on STEM in senior secondary schools. Research analyzing related publications on STEM in high schools has not provided sufficient information. So the researchers conducted a bibliometric analysis related to STEM in high schools. In characterizing a particular research area, bibliometric analysis methods are used in a systematic review process that detects research patterns and current issues from previous publications (Nandiyanto et al., 2023). Recently, many reports on bibliometric analysis have been published to understand research trends on certain topics (Muhammad, Himmawan, et al., 2023;Muhammad, Marchy, et al., 2022, 2023Muhammad, Mukhibin, et al., 2022).
Research related to this research is as conducted by Assefa (2013), Gil-Doménech, Berbegal-Mirabent, and Merigó (2020), Li et al. (2009), Lin andHo, (2015), and Özkaya (2019) regarding bibliometric analysis that includes STEM variables. In addition to bibliometric analysis, research related to this field is also discussed using the systematic method literature reviews (Winberg et al., 2019). These studies take data sources based on databases Web of Science has not yet tried to fetch the required data from the database scopus. For this reason, researchers conducted a bibliometric analysis related to STEM in high schools from 1988 to 2023.
The purpose of this research is to capture the STEM-related research landscape in high schools from 1988 to 2023. The research questions are: (1) how do publication trends and citation trends relate to STEM research in high schools?; (2) how is the distribution of publication mapping and inter-country relations in research related to STEM in high school?; (3) what is the novelty and focus of STEM research in high schools?

RESEARCH METHODS
This research uses a bibliometric analysis method. STEM-related publications in senior secondary schools retrieved from the database Scopus from 1988 to 2023 were analyzed using descriptive bibliometric analysis. The stages of data collection started from identification, screening, eligibility and inclusion (Moher et al., 2009).
Identification was conducted by entering keywords according to the research theme to be studied. In this study, research on STEM in high schools will be discussed. For this reason, the researcher entered the keywords "STEM" and "education" in the database Scopus. After entering the keywords, data obtained as many as 7746 publications according to the specified criteria. The data for 7746 publications is then examined whether there is duplicate data or not. Since there is no duplication of data, 7746 publications can proceed to the next stage.
Screening is carried out to select publications from the first stage, publications must meet the following criteria, namely publication in the form of articles in journals and conferences and publications in English. After screening, 657 articles Volume 15, Issue 1, July 2023 | 104 were discarded and did not proceed to the next stage. A total of 7,089 that meet the next criteria will be carried out the eligibility stage.

Figure 1. Data Collection Process
Eligibility is conducting feasibility, titles and abstracts of 7089 documents from the previous stage will be seen whether these documents meet the criteria, namely titles and abstracts containing STEM words and learning mathematics, after feasibility is carried out, 118 publications that meet the criteria are obtained which can be continued at the inclusion stage.
Data obtained from the database Scopus is then saved in 2 different formats, first we save it in CSV form and save it in RIS form, CSV data will be entered and analyzed with the help of VOSviewer while RIS data will be entered into Harzing's Publish or Perish Software. The trend of publication and the tendency of STEM research citations in high schools is seen from the year of publication, and the distribution of countries and country relations is seen from the author's country of origin which will then be seen for novelty and research focus will be analyzed with the help of the VOSviewer application. Software Microsoft Excel and Software Harzing's Publish or Perish. VOSviewer is used in displaying the visualization of relations between countries which is the focus of research. Harzing's software Publish or Perish is used to calculate h-index and g-index values and other data about citations and Microsoft Excel software is used to display the distribution of countries on a world map.

RESULTS AND DISCUSSION
The search results for data that have been refined through several stages amount to 118 publications. These publications will then be analyzed using bibliometric analysis with the help of various applications, starting from publication trends, citations, and countries, as well as the novelty and focus of research related to STEM in high schools from 1988 to 2023.

Publication Trends and Citation Trends
The trend of publications related to STEM research in high schools from 1988 to 2023 continues to increase every year, this can be seen from the trend line shown in Figure 2.  (Evans et al., 1988), which is a writer from the United States of America. The trend of STEMrelated publications in mathematics can be seen in Figure 2.
In table 1, trend Citations can be seen from the NCP value, the highest NCP value in 2020 is NCP=16, followed in 2019 with NCP=15, but when viewed from the total citations (TC) published in 2011 it has the highest number of citations among other years with 169 citations, means publications in 2011 have been the most influential year in this field. While the C/P and C/CP values in 2011 were the highest, this was because the 2011 publication had a large number of citations but a small number of publications. The citation trend can also be seen from the hindex and g-index values, it can be seen that the highest h-index and g-index values occurred in 2019.

Mapping the Distribution of Publications and Relations Between Countries
Publications related to STEM in high school from 1988 to 2023 have been written from various countries. There are 17 countries that have published articles in this area. For this reason, the researcher displays the distribution of these countries as follows.

Figure3. Geographic distribution of publications
In Figure 3, it can be seen that Indonesia has published 60 documents related to STEM in high school, this is the largest number of publications compared to other countries, the Philippines is in second place with 20 publications. When viewed by continent, all continents have published their documents except for the African continent, the Asian continent is the continent with the most publications, namely more than 100 publications, which is around 85 percent of the total STEM-related publications in high schools. Figure 4 is the relationship between countries in STEM research in high schools, it can be seen that the State of Indonesia has a lot of cooperation with other countries, namely as many as 11 links. This means that Indonesia is not only a country with the highest number of publications but also a country with a high level of cooperation compared to other countries.

Novelty and Research Focus
Research related to STEM in high schools from 1988 to 2023 is divided into 3 clusters, these clusters are displayed with the VOSviewer application which shows the focus of research in this field. related to STEM in high schools from 1988 to 2023. The first research focus is keywords with red circles consisting of 11 items. The keywords with the largest circle diameter are STEM, mathematics, technology, science, engineering, outcomes, and motivation, meaning that keywords such as outcomes and motivation are the focus of the first research together with STEM. The second research focus is the keywords with green circles consisting of 8 items. Critical keywords thinking and the 21 st century has a large circle compared to the others, meaning that this keyword is the focus of the second research. The focus of the last research is the keywords with blue circles consisting of 7 items, which have the biggest circle are the keywords problem and development, meaning problem and development become the focus of recent research in this field.

Figure 6. Overlays Visualization
In Figure 6, it can be seen that the color indicates the year the keyword was published, meaning that the blue keyword is an old theme while the yellow keyword is a new theme related to STEM research in senior high schools. The new theme for this field is critical thinking, 21 st century, PjBL, Covid and perception. The novelty related to STEM research in high schools can be seen from the connectedness between keywords. Researchers look at the keywords that are the focus of research and new themes in determining the novelty of this research. Keywords that become new themes such as critical thinking are not yet directly linked to other new thematic keywords like perception. The keywords that became the focus of the first research were motivation and outcome together with STEM in high school, it is not directly connected with the new keyword, namely Covid.

What are the Publication Trends and Citation Trends Regarding STEM Research in High Schools?
Publications related to STEM research on mathematics learning from 1988 to 2023 began to experience an increase in the number of publications from 2016 to 2021. This is in accordance with what was stated by Özkaya (2019) research related to STEM has developed very rapidly in recent years.
Citation trends related to STEM research in high schools from 1988 to 2023 can be seen from several values. The 2011 publication with the highest number of citations, namely 169 citations, the 2011 article which had the highest number of citations, was research conducted by DeCaro and Rittle-Johnson (2012) with the title "The impact of problem-based learning strategies on STEM knowledge integration and attitudes: An exploratory study in among high school students" which explores the effect of problem-based learning (PBL) strategies on high school students' attitudes towards learning integrated knowledge in science, technology, engineering, and mathematics (STEM). The 5 publications in 2011 which became the year with the publication with the highest number of citations are as follows. Author ( year ) Citation 1 (S.-J.  79 2 (Ma, 2011) 37 3 (S. J.  32 4 (S.-J.  18 5 (Tims et al., 2011) 3

How is the Distribution of Publication Mapping and Inter-Country Relations in Research Related to STEM in High School?
The distribution of STEM-related publication mapping in high schools from 1988 to 2023 is spread across all continents except the African continent. Indonesia is the most influential country in this field because Indonesia has the largest number of publications compared to other countries, and Indonesia also has a high level of relations between countries. This is in accordance with what was stated by Thu et al. (2021) that Indonesia is one of the most influential countries in STEM research in high schools which has been analyzed by bibliometric analysis. The difference is that the research covers all middle schools or "middle school" while in this study the focus was on high school, and the range of years taken was also slightly different, but the data sources taken were both from database scopus.

What is the Novelty and Research Focus Related to STEM in High School?
The focus of research related to STEM in senior high schools from 1988 to 2023 is divided into three parts, namely, 1) outcomes, and motivation; 2) critical thinking and 21 st centuries; 3) problems and development. The first research focus is outcomes, and motivation. Research conducted by Fong et al. (2021) discusses a person-centred approach to motivation across domains with academic and STEM-related outcomes. The research concluded that motivation plays an important role in STEM Education. The second research focus is critical thinking and the 21 st century. According to Harun (2020) in the world of education, STEM is needed in order to follow the needs of the 21 st century, namely to support learning that is supported by technology, and students have scientific literacy. The third research focus is problem and development. Research conducted by (Ramli et al., 2020)discusses the process of developing and validating newly developed STEM Teachers and ensuring students work together in problem solving activities. The three research focuses above can be used as a reference for future researchers who choose a similar theme. For this reason, it is necessary to carry out further research related to the three research focuses above.
Keywords that become new themes such as critical thinking in a manner together not connected directly with keywords with other new themes such as perception. It means, this can be utilized for researchers furthermore, for can connect between three keywords, that is critical thinking, perception, and STEM. Like see teacher's perception or student about STEM approach to enhancement ability think critical, for example other Possible researcher furthermore can focus his research to effect from STEM Education to ability think critical thinking and perceptions of STEM students. Keywords that are the focus of the first research such as motivation and outcomes together with STEM in secondary schools are not directly connected with the new keyword, namely Covid. That is, it is a novelty useful for further research.

CONCLUSION
From the results and discussion it can be concluded that STEM-related publications in high schools continue to increase every year, especially from 2016 to 2021. The 2011 publication has cited more than 160 citations, this number is more than any other year. Indonesia is the most influential country in this field. The focus of research related to this field is 1) outcomes and motivation; 2) critical thinking and 21 st centuries; 3) problems and development. The new theme for this field is critical thinking, 21 st century, PjBL, Covid, and perception. Keywords that become the new theme such as critical thinking are not yet directly linked to other new thematic keywords like perception. The keywords that were the focus of the first research, such as motivation and outcomes together with STEM in high schools, were not directly connected with the keywords that became the new theme, namely Covid. That is, this is a novelty that is useful for further research.