Random matrix model for the Nakagami-q (Hoyt) fading in multiple-input multiple-output (MIMO) communication channels with arbitrary number of transmitting and receiving antennas is considered. The joint probability density for the eigenvalues of H† H† (or {H H†), where H† is the channel matrix, is shown to correspond to the Laguerre crossover ensemble of random matrices and is given in terms of a Pfaffian. Exact expression for the marginal density of eigenvalues is obtained as a series consisting of associated Laguerre polynomials. This is used to study the effect of fading on the Shannon channel capacity. Exact expressions for higher order density correlation functions are also given which can be used to study the distribution of channel capacity. © 2010 IEEE.}, author_keywords={Channel capacity; Fading distributions; Hoyt distribution; Laguerre crossover ensemble; Multiple-input multiple-output (MIMO) channels; Nakagami-q distribution; Random matrices}, keywords={Hoyt distribution; Laguerre; Multiple-input multiple-output channels; Nakagami; Nakagami-q distribution; Random matrices, Distribution functions; Eigenvalues and eigenfunctions; Fading channels; Probability density function; Receiving antennas, Channel capacity}, funding_details={Council of Scientific and Industrial Research, IndiaCouncil of Scientific and Industrial Research, India, CSIR}, funding_text_1={Manuscript received August 03, 2009; revised December 08, 2009. Current version published April 21, 2010. S. Kumar was supported by CSIR India through a fellowship. The authors are with the School of Physical Sciences, Jawaharlal Nehru University, New Delhi 110067, India (e-mail: skumar.physics@gmail.com; ap0700@mail.jnu.ac.in). Communicated by A. L. Moustakas, Associate Editor for Communications. Digital Object Identifier 10.1109/TIT.2010.2044060}, references={Simon, M.K., Alouini, M.-S., (2000) Digital Communication Over Fading Channels: A Unified Approach to Performance Analysis, , Hoboken, NJ: Wiley; Simon, M.K., Alouini, M.-S., A unified approach to the performance analysis of digital communication over generalized fading channels (1998) Proc. IEEE, 86 (9), pp. 1860-1877. , Sep; Foschini, G.J., Gans, M.J., On limits of wireless communications in a fading environment when using multiple antennas (1998) Wireless Pers. Commun., 6 (2), pp. 311-335. , Mar; Telatar, I.E., Capacity of multi-antenna Gaussian channels (1999) Eur. Trans. Telecommun., 10 (6), pp. 585-596. , Nov; Wang, Z., Giannakis, G.B., Outage mutual information of spacetime MIMO channels (2004) IEEE Trans. Inf. Theory, 50 (4), pp. 657-662. , Apr; Verdú, S., Shamai, S., Spectral efficiency of CDMA with random spreading (1999) IEEE Trans. Inf. Theory, 45 (2-3 PT), pp. 622-640. , Mar; Rapajic, P.B., Popescu, D., Information capacity of a random signature multiple-input multiple-output channel (2000) IEEE Trans. Commun., 48 (8), pp. 1245-1248. , Aug; Shin, H., Lee, J.H., Closed form formulas for ergodic capacity of MIMO Rayleigh fading channels (2003) Proc. IEEE Int. Conf. Communications, pp. 2996-3000. , May; Cottatellucci, L., Debbah, M., On the capacity of MIMO rice channels (2004) Presented at the 42nd Annu. Allerton Conf. Communication, , Oct; Jayaweera, S.K., Poor, H.V., On the capacity of multiple-antenna systems in Rician fading (2005) IEEE Trans. Wireless Commun., 4 (3), pp. 1102-1111. , May; Kang, M., Alouini, M.-S., Capacity of MIMO Rician channels (2006) IEEE Trans. Wireless Commun., 5 (1), pp. 112-122. , Jan; Müller, R.R., A random matrix model of communication via antenna arrays (2002) IEEE Trans. Inf. Theory, 48 (9), pp. 2495-2506. , Sep; Smith, P.J., Roy, S., Shafi, M., Capacity of MIMO systems with semicorrelated flat fading (2003) IEEE Trans. Inf. Theory, 49 (10), pp. 2781-2788. , Oct; Chiani, M., Win, M.Z., Zanella, A., On the capacity of spatially correlated MIMO Rayleigh-fading channels (2003) IEEE Trans. Inf. Theory, 49 (10), pp. 2363-2371. , Oct; Simon, S.H., Moustakas, A.L., Eigenvalue density of correlated complex random Wishart matrices (2004) Phys. Rev. E, 69. , R, Jun, 065101; Simon, S.H., Moustakas, A.L., Marinelli, L., Capacity and character expansions: Moment-generating function and other exact results for MIMO correlated channels (2006) IEEE Trans. Inf. Theory, 52 (12), pp. 5336-5351. , Dec; Ghaderipoor, A., Tellambura, C., Noori, M., On the eigenvalue distribution of correlated MIMO channels by character expansion of groups (2008) Proc. IEEE Global Telecommunications Conf, pp. 1-5. , Dec; Marques, P.M., Abrantes, S.A., On the derivation of the exact, closed-form capacity formulas for receiver-sided correlated MIMO channels (2008) IEEE Trans. Inf. Theory, 54 (3), pp. 1139-1161. , Mar; Kang, M., Alouini, M.-S., Quadratic forms in complex Gaussian matrices and performance analysis of MIMO systems with co-channel interference (2004) IEEE Trans. Wireless Commun., 3 (2), pp. 418-431. , Mar; Ordôñez, L.G., Palomar, D.P., Fonollosa, J.R., Ordered eigenvalues of a general class of Hermitian random matrices and performance analysis of MIMO systems (2008) Proc. IEEE Conf. Communications, pp. 3846-3852. , May; Nakagami, M., The m-distribution-A general formula of intensity distribution of rapid fading (1960) Statistical Methods in Radio Wave Propagation, pp. 3-36. , New York: Pergamon; Müller, A., Speidel, J., Ergodic capacity and information outage probability of MIMO Nakagami-m keyhole channels with general branch parameters (2007) Proc. Wireless Communications and Networking Conf, pp. 2184-2189. , Mar; Fraidenraich, G., Lévêque, O., Cioffi, J.M., On the MIMO channel capacity for the Nakagami-m channel (2008) IEEE Trans. Inf. Theory, 54 (8), pp. 3752-3757. , Aug; Sagias, N.C., Karagiannidis, G.K., Gaussian class multivariate Weibull distributions: Theory and applications in fading channels (2005) IEEE Trans. Inf. Theory, 51 (10), pp. 3608-3619. , Oct; Hoyt, R.S., Probability functions for the modulus and angle of the normal complex variate (1947) Bell Syst. Tech. J., 26, pp. 318-359. , Apr; Fraidenraich, G., Lévêque, O., Cioffi, J.M., On the MIMO channel capacity for the dual and asymptotic cases over Hoyt channels (2007) IEEE Commun. Lett., 11 (1), pp. 31-33. , Jan; Chytil, B., The distribution of amplitude scintillation and the conversion of scintillation indices (1967) J. Atmos. Terr. Phys., 29, pp. 1175-1177. , Sep; Annamalai, A., Tellambura, C., Bhargava, V.K., Simple and accurate methods for the outage analysis in cellular mobile radio systems: A unified approach (2001) IEEE Trans. Commun., 49 (2), pp. 303-316. , Feb; Youssef, N., Wang, C.-X., Pätzold, M., A study on the second order statistics of Nakagami-Hoyt mobile fading channels (2005) IEEE Trans. Veh. Technol, 54 (7), pp. 1259-1265. , Jul; Smith, P.J., Shafi, M., On a Gaussian approximation to the capacity of wireless MIMOsystems (2002) Proc. IEEE Int. Conf. Communications, 1, pp. 406-410; Ge, H., Wong, K.D., Barton, M., Liberti, J.C., Statistical characterization of MIMO channel capacity (2002) Proc. IEEE Wireless Communications and Networking Conf, 2, pp. 789-793. , Mar; Kamath, M.A., Hughes, B.L., Yu, X., Gaussian approximations for the capacity of MIMO Rayleigh fading channels (2002) Proc. 36th Asilomar Conf. Signals, Systems and Computers, 1, pp. 614-618. , Nov; Zhao, Y., Zhao, M., Xiao, L., Wang, J., Analytical expression for the MIMO channel capacity (2006) Tsinghua Sci. Technol., 11 (3), pp. 271-277. , Jun; Chizhik, D., Foschini, G., Gans, M., Valenzela, R., Keyholes, correlations, and capacities of multielement transmit and receive antennas (2002) IEEE Trans. Wireless Commun., 1 (2), pp. 361-368. , Apr; Shin, H., Lee, J.H., Capacity of multiple-antenna fading channels: Spatial fading correlation, double scattering, and keyholes (2003) IEEE Trans. Inf. Theory, 49 (10), pp. 2636-2647. , Oct; Wishart, J.B., (1928), 20 A, pp. 32-52; Wilks, S.M., (1963) Mathematical Statistics, , Hoboken, NJ: Wiley; Edelman, A., (1989) Eigenvalues and Condition Numbers of Random Matrices, , Ph. D. dissertation, Dept. Math., Massachusetts Inst. Technol., Cambridge, MA; Mehta, M.L., (2004) Random Matrices, , New York: Academic; Pandey, A., Ghosh, S., Skew-orthogonal polynomials and universality of energy-level correlations (2001) Phys. Rev. Lett., 87 (2), p. 02410214. , Jun; Ghosh, S., Pandey, A., Skew-orthogonal polynomials and randommatrix ensembles (2002) Phys. Rev. E, 65 (1-21), p. 046221. , Apr; Dyson, F.J., Distribution of eigenvalues for a class of real symmetric matrices (1971) Rev. Mex. Fis., 20, pp. 231-237; Brody, T.A., Flores, J., French, J.B., Mello, P.A., Pandey, A., Wong, S.S.M., Random-matrix physics: Spectrum and strength fluctuations (1981) Rev. Mod. Phys., 53 (3), pp. 385-479; Kumar, S., Pandey, A., Universal spectral correlations in orthogonal-unitary and symplectic-unitary crossover ensembles of random matrices (2009) Phys. Rev. E, 79, p. 02621114. , Feb; Kumar, S., Pandey, A., Jacobi crossover ensembles of random matrices and statistics of transmission eigenvalues (2010) J. Math. Phys. A: Math. Theor., 43 (1-22), p. 085001. , Feb; Pandey, A., Brownian motion model of discrete spectra (1995) Chaos, Solitons, Fractals, 5 (7), pp. 1275-1285. , Jul; Marčenko, V.A., Pastur, L.A., Distribution of eigenvalues for some set of random matrices (1967) Math. USSR-Sbornik, 1, pp. 457-483; Bai, Z.D., Methodologies in spectral analysis of large dimensional random matrices (1999) Statist. Sinica, 9, pp. 611-677; Risken, H., (1989) The Fokker-Planck Equation: Methods of Solutions and Applications, , 2nd ed. New York: Springer-Verlag; Szego, G., (1975) Orthogonal Polynomials, , Providence, RI: AMS; Widom, H., On the relation between orthogonal, symplectic and unitary matrix ensembles (1999) J. Statist. Phys., 94 (3-4), pp. 347-363. , Feb}, correspondence_address1={Kumar, S.; School of Physical Sciences, , New Delhi 110067, India; email: skumar.physics@gmail.com}, issn={00189448}, coden={IETTA}, language={English}, abbrev_source_title={IEEE Trans. Inf. Theory}, document_type={Article}, source={Scopus},