Understanding defects, disorder and doping due to N implantation in ZnO is one of the most debated issues for the last few years. In the present work, a comprehensive investigation has been carried out using Raman, photoluminescence (PL) spectroscopy and grazing-incidence X-ray diffraction on 50 keV N ions implanted granular ZnO with different fluence (approximately up to 6.5\% atomic concentration) along with postimplantation annealing. Raman investigation suggests that 275, 510, 643, and 857 cm−1 modes are directly related to nitrogen. Additionally, VZn may have some role in stabilizing 275 cm−1 mode. The broadening (or tailing) of E2 low mode is related to vibration of distorted Zn sublattice, which may be a product of ion implantation generated defect cluster like VZn–VO. The distortion starts to reduce with annealing at elevated temperatures. Direct correlation between 555 cm−1 Raman mode and the tailing of E2 low mode has been found. More defect clustering is vivid from the reduced PL of the ZnO samples with increasing implantation fluence. So, tailing of E2 low Raman mode, increasing intensity of 555 cm−1 mode and nonradiative defect centers are of common origin. Both the ratios E1(2LO)/E1(LO) and E2 high/E1(LO) can be used as parameters to measure the defective nature of ZnO after ion implantation/irradiation. Low temperature PL (selected samples) suggests absence of shallow acceptor states, although negative thermal quenching above 175 K has been observed (implantation fluence 1 × 1016 ions/cm2 and annealed at 500°C) which can be a signature of deep acceptors. © 2019 John Wiley & Sons, Ltd.