According to the guideline of Rsubread, we're supposed to feed the unmapping files with txt format into the software. Is there any suggestion for the converting or can I just input the fastq files directly? Many thanks!

As Martin noted, you can use Rbowtie, but that is for the original bowtie aligner, which doesn't do gapped alignments. If you are doing RNA-Seq you probably want bowtie2, which does do gapped alignments. You can run bowtie2 on Windows, so that is probably the best bet, but you have to run it from the command line, not from within R.

Most aligners assume you are using some sort of Linux variant, so you are sort of hamstrung by the fact that you are on Windows. But Linux is free after all, and it's relatively simple to set up a dual-boot Ubuntu/Windows OS on your comp, so if you are serious that might be something to consider.

One thing about kallisto and sleuth (and salmon or sailfish and sleuth while we are at it). These packages are intended to make comparisons at the transcript level, rather than the gene level. Since part of the alignment process is to infer which transcript a read came from, there is additional uncertainty in your count measurement that you have to account for when fitting a model. This has two downsides. First, that additional uncertainty has a cost, which is reduction in power to detect differences. Second, you shouldn't use something like edgeR or DESeq2 for transcript-level counts because the model they fit doesn't account for that uncertainty, so you have to use something like sleuth (either Lior Pachter's version or the patched version from Rob Patro's group) to fit the model. And sleuth is just a github package now, so you are pretty much on your own if you want to go that route.

As an (apparent) beginner, you are probably better off just getting bowtie2 and going from there.
 

The Rbowtie package wraps (an older?) version of the Bowtie aligner, but probably most people use alignment tools outside R. The airway vignette and differential expression work flow describe overall approaches that go from FASTQ to count matrices via whole-genome alignment. kallisto is a different and fast though not cross-platform approach; see SummarizedExperiment::readKallisto() in addition to the github sleuth package. 

The poster has FASTQ files, but needs alignment (BAM) files before trying to count reads; b.nota's efforts would only be relevant after alignment. Ways to summarize aligned reads to counts across platforms and in R include bamsignals or perhaps GenomicFeatures::summarizeOverlaps().

I counted the reads once in R with a self made script using the libraries: IRanges, GenomicRanges, and Rsamtools. However, if you are pretty new to RNA-seq I would not recommend to try this yourself. It was pretty hard to do this.

I think the easiest way for you to get your counts is to install a virual machine with Ubuntu and try featureCount in Rsubread there.

Unfortunately is not available in windows-based R.

Rsubread has been available in R for Windows for a few years now.