Friction stir welding (FSW) is a new entrant in welding technology and getting a defect-free weld is the final objective. But different defects are generated due to various reasons and needs to be analyzed to eliminate them. The aim of the research work is to identify and classify different kinds of surface defects generally encountered during the FSW process using digital image processing techniques. The defects on the surface of the weld are identified using image pyramid and image reconstruction algorithms. Further, using these algorithms the defects can be classified into voids, grooves, cracks, key-hole and flash with the help of unique features of each kind of defect. Vertical intensity plot and the area plot of the defect blobs are represented for the proper localization and analysis of severity of defects. © 2016 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved.