Dom-Tree Based Automatic Classification Of Predatory Journals Using Doc2vec And Automated Machine Learning

Abstract

Predatory journals threaten academic integrity by offering publication without proper peer review. Indonesia ranked second globally, with 16.73% of articles suspected to have been published in predatory journals during 2015–2017. This study aims to develop a method for classifying the web pages of predatory journals using a combination of Distributed Representations of Documents (Doc2Vec) and Automated Machine Learning (AutoML) based on the structure of the Document Object Model (DOM) tree. The dataset of predatory journals was collected from Kaggle, while non-predatory journals were obtained from the Directory of Open Access Journals (DOAJ). The main pages of journal websites were collected through web scraping and converted into a DOM corpus using two traversal approaches: Depth-First Search (DFS) and Breadth-First Search (BFS). The DOM corpus was then vectorized using Doc2Vec and automatically classified with AutoML from Auto-Sklearn. The evaluation was conducted using accuracy and macro avg F1-score metrics for each traversal method and training time configuration. AutoML training was tested within a range of 15 to 120 minutes, in 15-minute intervals. The best model for BFS was obtained at 15 minutes of training with a macro avg F1-score of 0.7812 and an accuracy of 0.9196. Meanwhile, the best model for DFS was achieved at 90 minutes of training with a macro avg F1-score of 0.7853 and an accuracy of 0.9255. These results indicate that the traversal method used to construct the DOM corpus influences the performance of the predatory journal classification model. DFS tends to yield better performance than BFS in the context of Doc2Vec and AutoML based on the DOM tree structure, as reflected in both accuracy and macro avg F1-score.