We perform our DE analysis with limma+voom packages with clean reads without  PCR duplicates removal.

for testing how PCR duplicates would affect DE gene and FPKM value, we also map reads with PCR duplicates removal to our reference and perform DE analysis with the same pipeline.

It turns out that most of the DE genes are overlap in both PCR duplicates removal group and without PCR duplicates removal group.Still, there are some genes that are unique in both groups,which takes up 4%~5%.

So, how do I evalutate this 5% DE genes?should I give them up or retain both of them?

I wouldn't recommend doing duplicate removal for DE analyses. Genuinely distinct fragments that have exact overlaps (which is highly likely when you limit yourself to exonic regions) will be marked as PCR duplicates and removed. This will result in smaller counts and reduced power to detect differences between libraries. In fact, for high-coverage transcripts, duplicate removal will cap the count to be the same in all libraries, preventing you from detecting DE for those genes. Even worse, if your libraries are of different sequencing depths, then the same capped count across libraries will yield different normalised expression values. If this happens for a non-DE gene, you can end up detecting more false positives.

All in all, I would suggest retaining the reads corresponding to putative PCR duplicates. If you have enough replicates, then that will automatically protect you against detecting spurious DE due to the duplicates. This is because duplicates are stochastic - if you get a stack of duplicates in one replicate and not another, the variance will be inflated and you will (correctly) fail to detect that gene as DE.