What happens to the little bit of RNA that has been made by the RNA primer during DNA replication?

The RNA primer is removed by a different DNA polymerase. First, the DNA polymerase attached the first nucleotide of DNA to the RNA primer, and then continues to synthesize DNA. Then, a different DNA polymerase comes along and removes the RNA primer and replaces it with DNA nucleotides through a process known at 5' to 3' exonuclease activity.