I know... you have the doubt that a dirty floppy disk could dirty or even ruin your drive. And you're right! Here's how to clean 3.5" floppies.
Through this tool that RetroManiak gave me, we can control both sides of the magnetic disk: they must not have round stains. Those are moldy and can damage the drive.
If you find such stains, I would not use the floppy disk, unless there is extremely important data.
Then we clean both sides of the magnetic disk with a soft microfiber cloth. I recommend, gently! A cotton cloth can also be used. The important thing is to act gently! You just have to dust both sides of the magnetic disk, without rubbing too much. Here's how to clean 3.5" floppies!
You can now insert the floppy disk into your computer, without fear that dirt will ruin the drive.
The same procedure applies to 5.25" diskettes, but there is a different tool.