Pigeonpea (Cajanus cajan) has gained worldwide attention among forage legumes due to its important role in animal nutrition, as well as its resilience to drought and high temperatures, making it particularly valuable in the semiarid region of Brazil. However, its sensitivity to photoperiodism has driven breeding programs to develop cultivars that are insensitive to this condition. In this study, we evaluated the genetic diversity and structural genetic profile of F-4 generation pigeonpea accessions and explored the association of inter-simple sequence repeat (ISSR) markers with forage production traits under neutral photoperiod. Mixed linear models were used in order to control confounding factors and reduce errors. Markers successfully distinguished between genotypes, as shown by mean marker index and resolving power values of 1.57 and 2.86, respectively. Accessions displayed genetic diversity characteristic of their generation, with an effective number of alleles (Ne) of 1.23 and an average Shannon diversity index of 0.24. Bayesian analysis identified four genetic groupings contributing to the structure of the C. cajan collection, based on a posterior probability threshold of 0.7. Assuming significant effect, P-value < 0.05, 14 loci were associated with growth rate after 40 days (GR), height after flowering (HF), days to flowering (DF), total dry forage mass productivity (FP), and pod productivity (PP). After adjusting for the false discovery rate, 16 markers were identified, six of which showed likely pleiotropic effects. A 1500-bp fragment from primer UBC-845 was associated with HF, DF, and PP. These markers proved useful for characterizing the pigeonpea collection and for identifying markers linked to forage production traits under neutral photoperiod conditions.