In the documentation of scran::pairwiseWilcox, it says that:

The weight for the p-value in a particular level of block is defined as NxNy, where Nx and Ny are the number of cells in clusters X and Y, respectively, for that level.

This is reasonable but I was wondering if it's common practice in the literature. May I have a reference (or more) for this, please.

No.

As in, I don't have a reference. I wouldn't be surprised if someone has done it, though, it's a pretty obvious way to weight results from multiple Wilcoxon tests.

In general, all of the scran::pairwise* functions attempt to weight by a value that (i) depends on the number of cells in each block and (ii) is relevant to the variance of the effect size. The product of the number of cells is the appropriate choice for the Wilcoxon test as the AUC is computed by dividing the test statistic by the product, and a larger product will reduce the variance of the AUC.

Note that this also means that each test uses different weighting schemes.