Crystal plasticity finite element method is a useful tool to investigate the anisotropic mechanical behaviors as well as the microstructure evolution of metallic materials and it is widely used on single crystals and polycrystalline materials. However, grain boundary involved mechanisms are barely included in the polycrystalline models, and modeling the interaction between the dislocation and the grain boundaries in polycrystalline materials in a physically consisstent way is still a long-standing, unsolved problem. In our analysis, a dislocation density based crystal plasticity finite element model is proposed, and the interaction between the dislocation density and the grain boundaries is included in the model kinematically. The model is then applied to Al bicrystals under 10% compression to investigate the effects of grain boundary character, e.g. grain boundary misorientation and grain boundary normal, on the stress state and the microstructure evolution. The modeling results suggest a reasonable correspondence with the experimental result and the grain boundary character plays a crucial role in the stress concentration and dislocation patterning.