Abstract:
Systematic Literature Reviews (SLRs) are based on structured methodologies for synthesizing research, but manual taxonomy development faces significant scalability and objectivity challenges. This study introduces a generalized, semi-automated approach for developing and validating SLR thematic frameworks, evaluated through a comparative benchmarking of three distinct computational paradigms: Latent Dirichlet Allocation (LDA), Louvain network analysis, and BERTopic. Using explicit mapping criteria and Jaccard similarity coefficients, we assess each model's standalone performance against a manual expert baseline, demonstrating substantial conceptual overlap while delineating clear algorithmic boundaries. Beyond structural validation, we evaluate the models’ capacity to extract domain-specific business insights by benchmarking the computational outcomes against an established SLR on agile software development projects. This evaluation reveals quantifiable alignment in areas like Quality and Communication, alongside critical tensions regarding upfront Architecture & Design and Large-Scale implementations. Crucially, the benchmarking uncovers a significant semantic paradigm shift, revealing how traditional project constraints are dynamically reframed in Agile discourse; instead of explicit Budget or Schedule metrics, these concepts are absorbed into execution-level operational proxies like ‘effort’ and ’estimation’. Furthermore, while sparsely documented domains like Maintenance remain entirely undetected by traditional LDA, BERTopic uniquely surfaces the explicit maintenance concept alongside its technical sub-dimensions (e.g., refactoring). These findings highlight the respective strengths and limitations of each paradigm, offering empirical guidance for SLR automation and methodological advancement across diverse scientific domains. Beyond the immediate SE domain, this work contributes to the broader effort of structuring unstructured scientific knowledge into coherent, machine-interpretable conceptual schemes.