It is well known that the usefulness of a machinelearning model is due to its ability to generalize to unseendata. This study uses three popular cyberbullying datasets toexplore the effects of data, how it’s collected, and how it’slabeled, on the resulting machine learning models. The biasintroduced from differing definitions of cyberbullying and fromdata collection is discussed in detail. An emphasis is made onthe impact of dataset expansion methods, which utilize currentdata points to fetch and label new ones. Furthermore, explicittesting is performed to evaluate the ability of a model togeneralize to unseen datasets through cross-dataset evaluation.As hypothesized, the models have a significant drop in theMacro F1 Score, with an average drop of 0.222. As such, thisstudy effectively highlights the importance of dataset curationand cross-dataset testing for creating models with real-worldapplicability. The experiments and other code can be found athttps://github.com/rootdrew27/cyberbullying-ml.
