ANALYSIS OF STUDENTS’ MATHEMATICAL LITERACY SKILL IN SOLVING PISA MATHEMATICAL PROBLEMS

This study aims to determine students’ mathematical literacy skills in solving PISA mathematical problems. Mathematical literacy is the capacity of individuals to formulate, use, and interpret mathematics in various contexts. Students’ mathematical literacy skills can be seen in the way students work in solving PISA questions. This study is qualitative descriptive research. Data from the analysis of students’ mathematical literacy skills were obtained through tests in the form of descriptive questions adopted from PISA questions, analysis based on indicators of mathematical literacy skill according to PISA, and the interview results with students who worked on the PISA questions. This study shows that students’ mathematical literacy skills were in the medium to the low category; this can be seen from the percentage of students’ answer scores in working on PISA questions. Therefore, it is necessary to provide reasoning questions such as PISA questions to help students improve their mathematical literacy skills.


INTRODUCTION
athematical literacy skills in mathematics help humans understand the role and use of mathematics in life and make correct decisions as educated humans. In addition, humans can use this skill to solve problems related to various life contexts mathematically according to mathematical principles (Rizki & Priatna, 2019); therefore, students from elementary school to university level need to take mathematics (Newton & Tonelli, 2020;Rogers & Kosko, 2019;Sarama & Clements 2009;Stodolsky, 1988;Tatto, Peck, Schwille, Bankov, Senk, Rodriguez, Ingvarson, Reckase, & Rowley, 2012).
Mathematical literacy is still a major challenge in basic mathematics education, so improving literacy is a challenge worldwide since people need literacy skills to find, select, interpret, analyze, and produce relevant information (Genlott & Gronlund, 2016). Previous studies reveal that teachers have made every effort to improve the teaching and learning processes to improve students' mathematics learning achievement (As'ari, 2015). However, some things remain to be followed up, one of which is related to students' low mathematical literacy skills in learning mathematics (Ayalon & Wilkie, 2020;Hu, Gong, Lai, & Leung, 2018;Ketonen & Hotulainen, 2019).
Mathematical literacy is basically an individual's ability to formulate, employ, and interpret mathematics in various contexts. Mathematical literacy includes mathematical reasoning and using mathematical concepts, procedures, facts, and tools to describe, explain, and predict phenomena. Mathematical skills help individuals recognize that mathematics plays a role in every aspect of life and makes informed decisions needed in a constructive, engaged, and reflective manner (Colwell & Enderson, 2016;Genlott & Gronlund, 2016;Purpura, Schmitt, & Ganley, 2017).
The results of PISA research show that the level of Indonesian mathematical literacy is always in the bottom 5 position (Fitriyani & Mastur, 2017). The PISA report published by the Organization for Economic Cooperation and Development (OECD) in mathematical literacy, Indonesia was ranked 68th out of 74 PISA participating countries in 2009, 64th out of 65 PISA participating countries in 2012, and ranked 65th out of 72 PISA participating countries in 2015 with 386 points (Hayati & Kamid, 2019). In 2018, Indonesia achieved a score of 379 for mathematics, which decreased compared to the PISA results in 2015. This result is very concerning, considering that the ability of Indonesian students, especially in mathematics at the international level, is still very low (Asdarina & Ridha, 2020). The data collected show that the mathematical literacy skills of Indonesian students are low and far from expectations. In fact, mathematical literacy is an aspect that is considered important and used in problem-solving in student's real life (Fitriyani & Mastur, 2017;Hayati & Kamid, 2019;Syawahid, 2019).
Mathematical literacy refers to the ability of students to apply mathematical knowledge and skills acquired from the classroom to their reallife experiences and understand any situation involving mathematics. In addition, mathematical literacy includes the ability to consider 'when' and 'how' to apply mathematical knowledge. Mathematical literacy consists of two components, namely knowledge and competence. Knowledge refers to conceptual and procedural knowledge used to connect and solve mathematical problems faced in real life. Conceptual knowledge refers to knowledge of facts, meanings, constructs, ideas, principles, laws, formulas, and concepts on mathematical topics. Meanwhile, procedural knowledge refers to knowing how to use mathematical procedures, language and symbols, and interpret and draw graphs and tables. Competence refers to the ability of students to apply mathematical knowledge and skills acquired from the classroom to their reallife and to understand situations involving mathematics (Sumirattana, Makanong, & Thipkong, 2017).
OECD (2015) divides mathematical literacy into several dimensions, namely: (a) Content dimension includes space and shape, change and relationship, quantity and uncertainty; (b) Process dimension includes the reproduction of definitions and computation, connection and integration for problem-solving, and reflection on mathematical thinking, generalization, and understanding; and (c) Situational or contextual dimension includes personal, educational and occupational, community and science or intra-mathematics. Furthermore, OECD (2017) states the mathematical processes that become the basis for analyzing students' mathematical literacy skills are: (a) modeling the problem into mathematical form; (b) applying mathematical concepts, facts, procedures, and reasoning; and (c) interpreting, applying, and evaluating the results obtained. In addition, in developing the PISA theory, there are seven basic mathematical skills that are the mainstay of the mathematical literacy process: (a) Communication, mathematical literacy involves communication. This is because, when a problem is solved, those who solve the problem need to express and explain the results of their thoughts to others so that other people can also understand the results of their thoughts; (b) Mathematics, mathematical literacy skills also involve abilities in mathematics, namely the ability to translate everyday language into mathematical forms, either in the form of concepts, structures, making assumptions or modeling mathematics; (c) Representation, it is the ability to represent mathematical objects such as graphs, tables, diagrams, pictures, equations, formulas, and other concrete forms; (d) Reasoning and arguments, reasoning and argumentation abilities are the roots of logical thinking processes developed to find a conclusion that can justify the solution to a problem; (e) Designing strategies to solve problems, this ability is related to a person's ability to use mathematics to solve the problems at hand; (f) The use of symbols, formal and technical language, and the use of operations, this ability involves understanding, interpreting, the ability to manipulate a mathematical context used in solving related problems; (g) The use of mathematical tools, it is the ability to be able to use various kinds of tools that can help the process of saving and knowing these limitations.
Mathematical literacy consists of 6 levels (OECD, 2015). The higher the level, the more complex the problem is. The six levels are presented in the form of PISA questions. The description of PISA questions based on the level of students' mathematical literacy skills is shown in the following table. Using knowledge to solve routine problems and solving general context problems 2 Interpreting problems and solving with formulas 3 Carrying out procedures properly in solving problems and choosing problem-solving strategies 4 Working effectively with models and selecting and integrating different representations, then relating them to the real world 5 Working with models for complex situations and solving complex problems 6 Using reasoning in solving mathematical problems, making a generalization, formulating, and communicating the findings.
PISA questions are used to determine students' mathematical literacy abilities since PISA is an international study that focuses on students' mathematical literacy skills aged 15 years. The PISA questions are created containing mathematical processes that aim to determine the students' mathematical literacy skills trying to solve the problems. For this reason, it is necessary to provide tests to measure students' mathematical literacy skills. Referring to the background mentioned above, the researcher raised the topic of analyzing students' mathematical literacy skills in solving PISA mathematics problems.

METHODS
This study is descriptive research with a qualitative approach. This research type is used to describe students' mathematical literacy abilities based on tests in the form of descriptive questions adopted from PISA questions, analysis based on indicators of mathematical literacy skill according to PISA, and the results of the interviews with students who worked on the PISA questions. The analysis results refer to the mathematical literacy process, which is an indicator in analyzing mathematical literacy skills. The indicators that become the basis for analyzing mathematical literacy skills according to PISA theory are: (a) Modeling questions into mathematical form; (b) Applying and formulating mathematical concepts, facts, procedures, and reasoning; and (c) Interpreting, applying, and evaluating the results obtained. For deeper analysis, data was also obtained from the results of interviews about the difficulties and what students faced when working on mathematics problems (Hu, Gong, Lai, & Leung, 2018;OECD, 2006;She, Stacey, & Schmidt, 2018) The research subjects are six students of Grade VII at SMP Negeri 1 Sinjai Selatan, Sinjai Regency, South Sulawesi Province. The purposive sampling technique is used to select the subjects based on the considerations and objectives of the researcher. PISA is an examination system initiated by the Organization for Economic Cooperation and Development (OECD) to evaluate the education system for children or students aged 15 years; therefore, the researcher's consideration is that Grade VII students are students with an average age of 15 years.

RESULTS AND DISCUSSION
This research was carried out on 6 students of Grade VII selected by their mathematics teacher since they can communicate their thought, either spoken or written well. Ojose (2013) stated that most people have difficulty working on literacy questions at levels 4, 5, and 6. Therefore, the researcher tested the test instrument in the form of test questions on the description of mathematical literacy skills, which consisted of 3 question items, namely level 4 in question number 1, level 5 in question number 2, and level 6 in question number 3. The results of the instrument test about the students' mathematical literacy abilities are shown in the following table.  Table 2 shows the results obtained by students in completing test questions for mathematical literacy skills level 4, 5, and 6. Level 4 shows that the student percentage only reached an average of 22.22%, which can be categorized as low, meaning that some students have not been able to represent concrete but complex real-life situations in mathematical models. Furthermore, Level 5 shows that the percentage of students reached 50%, meaning that students could understand more complex mathematical models and were sufficiently able to solve more complicated situations in mathematical models. Finally, Level 6 shows the same percentage as question number 1. The score of each question item shows that no student could show maximum results, indicating that students have not been able to do good reasoning in understanding the mathematics problems.
Furthermore, table 2 illustrates that the mathematical literacy skills of Grade VII students are still far from expectations, so special attention is needed about the reasons why students' mathematical literacy skills are low. It is expected that students' mathematical literacy skills can increase according to the standard for assessing students' mathematical literacy skills. Ojose (2013) states that many factors cause such situations, namely home, school, parents, society, etc. Therefore, it can be seen from a student's perspective based on the interview results about the difficulties and what students faced when working on mathematics problems.
Students' low mathematical literacy skills are triggered by the number of students who have difficulty working on mathematical literacy skill questions, as reflected in the student statements during interviews. The following is an analysis of the results of the students' mathematical literacy skills test:

Level 4
Level 4 question: A pizza seller provides two different pizza sizes but with the same taste and thickness. The small pizza is 20 cm in diameter and is sold for IDR 20,000.00, and the large one is 40 cm in diameter and is sold for IDR 40,000.00. Which pizza sale is more profitable for the seller?
There are three kinds of answers given by students. Some answered that pizza with a diameter of 40 cm, some answered that both pizzas were profitable, and some answered that pizza with a diameter of 20 cm was more profitable.
Here is the work of students who answered the pizza with 40 cm in diameter. "A pizza with a diameter of 40 cm sold for IDR 40,000 is more profitable since customers can enjoy a pizza with a larger size and the same taste and thickness as a pizza with a diameter of 20 cm." The initial request for solving this problem is to find the price per unit area of each pizza. Meanwhile, finding the idea of solving this problem requires good reasoning and analytical skills.
From the answers above, the student wrote that "customers can enjoy a larger size pizza." In fact, the question is which pizza benefits the seller instead of the consumer. Thus, it can be said that the student is weak in mathematical ability or translating everyday language into mathematical form, whether it is a concept, a structure, making assumptions, or modeling mathematics. The answer above shows that students could not analyze the question properly. During the interview, students who answered the question like the one above said they thought what they wanted to look for was pizza which was more satisfying to customers, and they also said there might be errors when reading the questions. These interview results indicate that the students could not understand the contents and intent of the questions correctly, causing them to be unable to model problems into mathematical form correctly and formulate or conduct procedures properly. However, students could also experience reading errors since the question is clear that what is being asked is pizza that benefits the seller. A study by Wutsqa (2017) revealed that most students have difficulty translating the problems posed. This is due to the characteristics of mathematical literacy questions that adopt PISA questions containing context in each problem, while on the other hand, students are accustomed to solving questions with closed characteristics and are routine in nature. In addition, a study by Fakhriyana, Mardiyana, and Aryuna (2018) stated that students who are not always able to understand mathematics problems can be caused by students' low logical ability. In addition, Akbar, Diniyah, Akbar, Nurjaman, and Bernard (2018) said that students always find it difficult to do mathematics problems because they think mathematics is not interesting, difficult, monotonous, and there are too many formulas that are difficult to memorize.
The following is the student's answer stating that the sale of the two pizzas is equally profitable: Figure 2. The Work of Students who Answered Both Pizzas was Profitable "Neither of the two pizzas is the most profitable since the sales value is the same if the pizza diameters are the same: (1) We make a pizza with a diameter of 120 cm. Thus, we make 6 pizzas with a diameter of 20 cm, and the price is 6 x IDR 20,000 = IDR 120,000; (2) 3 pizzas with a diameter of 40 cm are made, the price is 3 x IDR 40,000 = IDR 120,000." The student answered, "the two pizzas are not the most profitable because the sales value is the same," meaning that students understood the meaning of the problem but were unable to formulate, apply concepts and procedures to work on the mathematics appropriately. Students only compared sales prices without any process of modeling and applying the concept first. This result is in line with a study by Muslimin and Sunardi (2019), stating that students have not been able to manipulate the questions into formulas used to solve the problems in the questions. The results of this analysis are actually different from the research conducted by Wati, Sugiyanti, and Muhtarom (2019), who found that the process of students' mathematical literacy skills was classified as good until the stage of formulating problems mathematically. This is also confirmed in the students' interview results who answered questions, as in figure 2. They said that this method was the only thing they thought of at the time and did not think that there were other procedures in answering the question. In addition, other students said that they were accustomed to working on questions that have explicit formulas (had been taught during the learning process) so that they were accustomed to working on questions that only needed to enter numbers in mathematical formulas. This result indicates that reasoning ability and creative thinking are also very influential on students' mathematical literacy skills. Masjaya and Wardono (2018) found that students are generally not allowed to take the initiative, seek their own answers, and formulate arguments. Students are generally faced with the question of how to solve the problem, not why the solution is so.
The following is the student's answer stating that the 20 cm pizza was more profitable: shows that students were considered to have been able to answer the questions given and also be able to provide strong reasons for the answers. Few students were able to answer correctly according to the procedure for working on the questions. When the student was asked about the question above, the student answered that he/she had never seen or encountered the same or similar questions before, but he/she understood that the problem was about the area of a circular shape. For students who answered question number one with the work steps above, it can be said that they have good mathematical literacy skills. This is because they understood the meaning of the questions and what must be done to answer these questions, meaning that they could represent the questions well. Choosing to find the area of the two pizzas is a description of the student's ability to model the problem into mathematical form.
Furthermore, students were considered to have a good ability in reasoning seen from the selection of work procedures before concluding that they first looked for the price per unit of each pizza. This process measured students' reasoning ability. Looking for the price per unit of the two pizzas is an initial idea. If students reach this stage, it can be said that they have good reasoning since by knowing the price per unit of each pizza, they can conclude that 20 cm pizza was more profitable. This result is in line with Hayati and Kamid (2019) study stating that the essential part of mathematical literacy is the mathematical process, i.e., the formulating process, using and interpreting, and evaluating mathematics from various contexts.

Level 5
Level 5 question: The nets of a cube-shaped cardboard can be made by opening the edges of the cardboard. The arrows in the image below show the direction of cutting the edges of a cardboard. Draw a grid of cardboard reflecting the image below. The question is about space and shape. Only a few students were able to answer this question.
Here is the answer of students who could draw correctly. Question number 2 is considered difficult for students. Only two students could answer this spatial question, and the rest four students were unable to answer correctly. Students who could answer this question are considered to have good spatial abilities. Related to mathematical literacy skills, this mathematics problem does not require students to design mathematical models rather than emphasizes spatial abilities, where students were asked to imagine the results of cubes. Based on the theory of basic mathematical literacy skills, this problem is more directed on the ability to represent questions with the help of pictures. If students can answer this question correctly, it can be said that students' literacy skills are good in the change and relationship categories. Students who could not answer this question confirmed in the interview that "I did not know which piece to start from and open in which direction." This result indicates that the students could not understand the instructions on the question requiring them to follow the direction of the arrow so that the students were unable to work on the question at all. Viewed from the basic mathematical skills of students at level four, students were classified as weak in representational abilities. The representation ability is the ability to represent mathematical objects such as graphs, tables, diagrams, pictures, equations, formulas, and other concrete forms. This result is in line with research conducted by Munfarikhatin, Natsir, and Merauke (2020), stating that some students have not been able to use their mathematical skills to solve PISA mathematics problems related to change and relationship contents. Meanwhile, Hasibuan, Fauzi, and Mukhtar (2019) stated that students' mathematical literacy skills in change and relationship contents are generally above average and classified as quite good. This is different from the results of this study shows that most students find it difficult to answer correctly. There are many difficulties experienced by students when solving the PISA questions given. Students expressed that there were difficulties they faced while answering the questions. One student said he/she did not understand the meaning of the question, other students felt anxious and confused when they saw questions that were considered difficult, so that they were not able to reason or think well to solve the problem.

Level 6
Level 6 question: A glass manufacturer produces glasses of the same shape and size. However, it turns out that there is 1 glass made of material A mixed with 999 glasses made of material B. The glass made of material A has a lighter weight than the glass made of material B. The manufacturer only has 1 scale that is able to weigh the most 700 cups with an accuracy level of up to milligrams. Determine the minimum amount of weighing to obtain 1 glass made of material A?
No student was able to find the correct answer. Although some of them could find the initial idea, they failed in continuing the procedure properly.
Here is the student's almost correct answer: Figure 6. The Work of Student for Question Number 3 "A total of 1000 glasses. Since the scales can only accommodate 700 glasses, the glasses are divided into two parts, 500 for each. Then, the glass is weighed, the lightest weight is glass A. Thus, there are 2 weighings." This question is the highest indicator category in PISA. The demand of this question involves mathematical literacy skills in terms of mathematizing, designing strategies, reasoning, and interpreting. In Figure 6, a student answered, "the glass is divided into two parts, namely 500 and 500." This answer indicates that students already have the initial idea of dividing the same number of existing glasses and weighing them but failed to proceed to the next process, i.e., formulating and carrying out mathematical procedures to get the correct answer. The student's answer concluded that "so, there were two weighings." This indicates that his/her mathematical literacy skills were still lacking because he/she only involved interpretive skills in answering questions, namely by dividing the number of glasses in half and weighing them. Kurniawati & Kurniasari (2019) stated that this happens because the ability to execute strategies is still relatively lacking. Students have not been able to continue the initial steps taken to find the correct answer.
Even students found it different to do basic things in mathematics, i.e., counting. They did not use the correct procedure in solving the mathematics problem since they merely guessed the answers. This matter is in line with Kiswanto, Kenedi, & Helsa (2017) study, stating that several variables can determine student literacy. In general, these factors can be grouped into two categories, namely internal and external factors. Internal factors include cognitive aspects such as intellectual abilities, numerical abilities, and verbal abilities, while non-cognitive aspects include interest and motivation. The external factors include the family environment, school environment, mass media, and social environment.
It is confirmed with the results of interviews showing that the difficulties experienced by students include difficulties in calculations, understanding questions, and analyzing questions. Furthermore, the higher the problem difficulty level, the higher the percentage of difficulty in analyzing the questions. This indicates students are not familiar with the questions that require reasoning in solving them rather than getting used to practical calculation problems. In addition, viewed from the type of questions, students find it serious difficulty when they have to answer story questions or questions about problem-solving in the context of everyday life. Thus, students need to be accustomed to dealing with real-world problems. This is in line with a study by Sumirattana, Makanong, & Thipkong (2017), stating that using a collaborative model between realistic mathematics and problem-solving can improve students' literacy or mathematical literacy skills.
Referring to the results and discussion of this study and previous supporting research, students' mathematical literacy skills can be improved by getting students to work on questions that require high-order thinking or reasoning. In addition, it is better to include questions with everyday contexts that emphasize students' reasoning abilities to make them get used to digging around mathematical problems. Thus, the ability to analyze and reason, as well as students' mathematical literacy skills can improve and comply with the expected standards.

CONCLUSION
The results of this study show that students' mathematical literacy skills are still in the low category. This is because students found it difficult to understand and analyzing questions as well as in representing questions in the form of mathematics, and students are not used to being faced with PISA standard questions. Therefore, schools need to present a more contextual and realistic mathematics learning and provide reasoning questions such as PISA standard questions, which are more helpful for students to improve students' mathematical literacy skills.