The relationship between brain structure and function is critical for revealing the pathogenesis of brain disorders, including Alzheimer’s disease (AD). However, mapping brain structure to function connections is a very challenging task. In this work, a bidirectional graph generative adversarial network (BG-GAN) is proposed to represent brain structure-function connections. Specifically, by designing a module incorporating inner graph convolution network (InnerGCN), the generators of BG-GAN can employ features of direct and indirect brain regions to learn the mapping function between the structural domain and the functional domain. Besides, a new module named Balancer is designed to counterpoise the optimization between generators and discriminators. By introducing the Balancer into BG-GAN, both the structural generator and functional generator can not only alleviate the issue of mode collapse but also learn complementarity of structural and functional features. Experimental results using Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset show that both generated structure and function connections can improve the identification accuracy of AD. The experimental findings suggest that the relationship between brain structure and function is not a complete one-to-one correspondence. They also suggest that brain structure is the basis of brain function, and the strong structural connections are majorly accompanied by strong functional connections.