We present a statistical analysis of the relative orientation between the plane-of-sky magnetic field and the filaments associated with the Galactic Cold Clumps. We separated polarization parameters components of the filaments and their background using thin optical medium assumption, the filaments were detected using the Rolling Hough Transform algorithm and we separated the clump and the filament contributions in our maps. We found that in high column density environments the magnetic fields inside the filaments and in the background are less likely to be aligned with each other. This suggests a decoupling between the inner and background magnetic fields at some stage of filaments’ evolution. A preferential alignment between the filaments and their inferred magnetic fields is observed in the whole selection if the clumps’ contribution is subtracted. Interestingly, a bimodal distribution of relative orientation is observed between the filamentary structures of the clumps and the filaments’ magnetic field. Similar results are seen in a subsample of nearby filaments. The relative orientation clearly shows a transition from parallel to no preferential and perpendicular alignment depending on the volume densities of both clumps and filaments. Our results confirm a strong interplay between the magnetic field and filamentary structures during their formation and evolutionary process.