The complex detection background and lesion features make the automatic detection of dermoscopy image lesions face many challenges. The previous solutions mainly focus on using larger and more complex models to improve the accuracy of detection, there is a lack of research on significant intra-class differences and inter class similarity of lesion features. At the same time, the larger model size also brings challenges to further algorithm application. In this project, we proposed a lightweight skin cancer recognition model with feature discrimination based on fine grained classification principle. The propose model includes two common feature extraction modules of lesion classification network and a feature discrimination network. Firstly, two sets of training samples (positive and negative sample pairs) are input into the feature extraction module (Lightweight CNN) of the recognition model. Then, two sets of feature vectors output from the feature extraction module are used to train the two classification networks and feature discrimination networks of the recognition model at the same time, and the model fusion strategy is applied to further improve the performance of the model, the proposed recognition method can extract more discriminative lesion features and improve the recognition performance of the model in a small amount of model parameters; In addition, based on the feature extraction module of the proposed recognition model, U-Net architecture, and migration training strategy, we build a lightweight semantic segmentation model of lesion area of dermoscopy image, which can achieve high precision lesion area segmentation end-to-end without complicated image preprocessing operation; The performance of our approach was appraised through widespread experiments comparative and feature visualization analysis, the outcome indicates that the proposed method has better performance than the start of the art deep learning based approach on the ISBI 2016 skin lesion analysis towards melanoma detection challenge dataset. This project is implemented with MATLAB software.