We present a brief account of the recent progress in the theoretical understanding of the electronic and structural properties of clusters of metals and semiconductors from ab initio calculations. The origin of the recently observed permanent electric dipoles in Nb clusters, the occurrence of magnetism in clusters of non-magnetic elements such as Pd, Rh and Ru, as well as the findings of the metal encapsulated clusters of Si, Ge, Sn and Pb are discussed. Empty caged clusters of Si, Ge and Sn have also been shown to be stable with H capping. These can be functionalized by exohedral doping for different applications while endohedral doping of the cages can be used to tailor highest occupied-lowest unoccupied molecular orbital gaps as well as the magnetic properties. Assemblies of such clusters could lead to novel nanostructures and new phases of these materials. {\textcopyright} 2005 Elsevier B.V. All rights reserved.