Collocational congruency relates to the degree of form-meaning overlap between an L2 collocation and its L1 translation. Congruency effect research consistently shows that low-proficiency language learners respond more slowly and less accurately than their higher-proficiency counterparts to collocations that are incongruent between the L1 and L2, rendering congruency an important quality for applied psycholinguistic collocation research. However, determining whether a given collocation is congruent is typically a binary decision based on expert judgment, which raises concerns regarding subjectivity and consistency. This study presents a data re-analysis of Wolter and Yamashita (2018) to compare expert judgment with two alternative continuous measures of collocational congruency that account for some collocations being “more congruent” than others: corpus-derived frequency and norming study data. The results indicate that while expert judgment is necessary, it is insufficient for reliably defining collocational congruency due to misclassification and multicollinearity issues. Terminological inconsistency was also identified regarding the precise definition of collocational congruency. Crucially, corpus-derived frequency emerged as a more effective complement to expert judgment than norming study data, the latter being resource-intensive and lacking adequate psychometric properties. From these findings, a set of best practice suggestions emerged to assist collocation researchers in identifying congruent and incongruent items.