The unsaturated granular material (UGM) is found to exhibit the moisture-sensitive and stress-dependent nonlinear cross-anisotropic behaviour in flexible pavements. This paper aims at developing a finite element (FE) model for pavement structure, which takes into account this behaviour of UGM. First, the Lytton model is employed to characterize the moisture-sensitive and stress-dependent behaviour of UGM, which incorporated a matric suction term to the existing stress-dependent constitutive model. The Lytton model is validated by the laboratory resilient modulus tests on the selected UGMs at different moisture contents. Second, the nonlinear cross-anisotropic constitutive equation of UGM is derived from the generalized Hooke's Law. The coefficients of the constitutive model are determined by the rapid triaxial test. Third, a User-Defined Material (UMAT) subroutine is developed to characterize this constitutive behaviour in the FE software ABAQUS. The UMAT subroutine adopts the secant stiffness approach with multiple damping factors. The UMAT subroutine is then implemented in the FE model of flexible pavement structures. The FE simulation results indicate the nonlinear cross-anisotropic model predicts greater pavement responses than the isotropic model. When the UGM is suction sensitive, it is found that the moisture content of UGM significantly affects the moduli distribution of base layer and the critical strains (i.e., tensile strain at the bottom of asphalt concrete, and compressive strains in base and subgrade layers) of pavement structures.